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WHERE REQUIREMENTS COME FROM 


NEW CAPABILITIES 
IMPROVED CAPABILITIES 
PROBLEM RESOLUTION 
IMPROVED EFFICIENCY 
INTERFACE COMPATIBILITY 
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REQUIREMENTS DEFINITION PROCESSES 


STATE THE PROBLEM 

DEFINE THE PROPERTIES OF A SOLUTION 
RESOLVE CONTRADICTIONS & INCONSISTENCIES 
ESTABLISH DESIRED LEVELS OF PERFORMANCE 
ORGANIZE, AMPLIFY & CLARIFY 
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Concepts of Usage 


WHO DOES IT SERVE? 

HOW DO THEY WORK? 

HOW DOES IT FIT IN? 

HOW DO WE THINK THEY WILL USE IT? 
WHAT SUPPORT DOES IT NEED? 

WHO WILL PROVIDE THAT SUPPORT? 
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TATINTL 


BUDGETARY COST ESTIMATES 


~CONTROLLABLE ELEMENTS 
Requirements 
Design 
Manufacturing 


TASKS 
Test 
Documentation 
Lifecycle Support 


UNCONTROLLABLE ELEMENTS 
Inflation 
Exchange Rates 
Stretch-Out 
Mission Urgency 


CONTINGENCIES 
Requirements Growth or Change 
Technical Risk 
Staffing Delays 
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BTATINTL | 
REQUIREMENTS NEGOTIATION 


FUNCTIONAL SCOPE 
PERFORMANCE LEVELS 
PHASING OF DELIVERY 
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STATINTL 


BUDGETARY COST ESTIMATES 


~ CONTROLLABLE ELEMENTS 
Requirements 
Design 
Manufacturing 


TASKS 
Test 
Documentation 
Lifecycle Support 


UNCONTROLLABLE ELEMENTS 
Inflation 
Exchange Rates 
Stretch-Out 
Mission Urgency 


CONTINGENCIES 
Requirements Growth or Change 
Technical Risk 
Staffing Delays 
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STATINTL 
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DEVELOPMENT OPTIONS 


BUY AN EXISTING PRODUCT 
DESIGN AND MANUFACTURE A “KNOWN QUANTITY” 
PLAN FOR A BREAKTHROUGH DURING DEVELOPMENT 
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STATINTL [ 
| CONTRACT OPTIONS 


“TYPE” 

DELIVERABLES 

DURATION AND SCHEDULE 
PAYMENT TERMS 
PERFORMANCE 

CLAUSES 
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_ CHOOSING A CONTRACTOR 


FIND QUALIFIED SOURCES 

EXPLAIN YOUR PROBLEM AND NEED 

COLLECT DATA FROM POTENTIAL CONTRACTORS 
EVALUATE 

NEGOTIATE 

OBTAIN A CONTRACT 

GO TO WORK! 
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MANAGING THE DEVELOPMENT PROCESS 


ROLES 


REQUIREMENTS MANAGEMENT 
COST CONTROL 

SCHEDULE MANAGEMENT 
TECHNICAL RISK MANAGEMENT 


TOOLS 
CONFIGURATION MANAGEMENT 
FINANCIAL REPORTS AND EVALUATION 


SCHEDULE TRACKING SYSTEMS 
TECHNICAL AUDITS AND REVIEWS 
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BUYING AND SELLING THE PRODUCT 


BUYING 


TESTING 
EVALUATION 
AUDITS 


SELLING 
BELIEVING 
EDUCATING 


CONVINCING 
TRAINING 
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TATINTL 
COTR Activities 
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DEVELOPING REQUIREMENTS 


UNDERSTANDING USER’S NEED 
RECOGNIZING TECHNICAL RISK 
NEGOTIATING BUDGETARY LIMIT 
TRADING NEED WITH TIME & BUDGET 
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STATINTL | 
DEFINING GOOD REQUIREMENTS 


DECOMPOSE 


WHAT ARE MY AIMS? 

WHAT ARE SOME MEASURABLE GOALS? 

WHAT OBJECTIVES MUST BE MET ALONG THE WAY TO THE GOALS? 
WHAT EXISTS? 

WHAT'S MISSING? 


IDENTIFY LIMITING FACTORS 


COST 
TECHNOLOGY 
INSTITUTIONAL 
PERSONNEL 
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PREPARING A CONTRACT 


EVALUATE TECHNICAL RISK 

FORMULATE FEE-BASIS RECOMMENDATION 

ESTABLISH FEE SCHEDULE 

DEFINE DELIVERABLES AND SCHEDULE 

ESTABLISH REPORTING AND DOCUMENTATION REQUIREMENTS 
COORDINATE WITH SECURITY, LOGISTICS, TRAINING, & FINANCE 
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STATINTL 
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QUALIFYING POTENTIAL CONTRACTORS 


INDUSTRY CONTACTS & PROFESSIONAL SOCIETIES 
QUALIFICATION BRIEFINGS 
BIDDERS CONFERENCE 
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STATINTL [ 
CHOOSING A CONTRACTOR 


CONSULTANTS 

STUDIES & ANALYSIS 
SPECIALTY DEVELOPMENTS 
SYSTEMS 
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STATINTL [ 
MANAGING THE CONTRACT PERFORMANCE 


MAINTAIN USER LIAISON 
DETERMINE TECHNICAL PROGRESS 
MANAGE REQUIREMENTS RISK 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


TATINTL LT 
CONTROLLING COST & SCHEDULE 


ACTIVELY MANAGE CRITICAL PATH ELEMENTS 
MINIMIZE WASTED EFFORT | 

ASSURE CLOSURE OF INTERFACES 
CONFIRMING ACTUAL PROGRESS 
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STATINTL 
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ADMINISTERING COMPLIANCE 


SECURITY 
GOVT. PROPERTY 
REPORTING 
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STATINTL 


MINIMIZING TECHNICAL RISK 


ENCOURAGE EARLY PROTOTYPING 
DEMAND OPTIONS FOR CRITICAL ITEMS 
SEEK BACKUP SOURCES 

MANAGE REQUIREMENTS 
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TATINTL 


ACCEPTING THE END PRODUCT 


AGREE TO TEST PLAN 
COORDINATE USER INVOLVEMENT 
REVIEW TEST RESULTS 

AUDIT DELIVERIES 
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STATINTL 
COORDINATING OPERATIONAL TURNOVER 


PROVIDE CONTRACT COVERAGE | 
IDENTIFY NEEDED SUPPORT OF OTHER ORGANIZATIONS 
ASSURE ACKNOWLEDGEMENT OF ROLES BY USER ORGANIZATION 
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STATINTL 


PRINCIPLES OF REQUIREMENTS 
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TATINTL 


WHAT CONSTITUTES A REQUIREMENT? 


CAPABILITIES 


FUNCTIONALITY 
MEASURABLE PERFORMANCE 
BOUNDARY CONDITIONS 


PROBLEM RESOLUTION 


THREATS 
DISRUPTION 
DISFUNCTION 
OPERATING COSTS 


COST CAP 


DEVELOPMENT 
OPERATING 
ANNUAL 


OPERATIONAL DATE 


INITIAL CAPABILIT 


owinei 6 of 


MPLETE CA onv8dcrdaboYe0001-9 
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STATINTL 
PROPERTIES OF REQUIREMENTS 


COMPLETE 


SCOPE OF NEED 
LIFE CYCLE SUPPORT 
PERFORMANCE 


CLEAR 


TERMS DEFINED 
SIMPLE STATEMENTS 


UNAMBIGUOUS 


SIMPLE CONDITIONS 
SPECIFIC 


TESTABLE 


EXPLICIT 

STIPULATED ENVIRONMENT 
KNOWN INTERFACES 
PREDICTABLE BEHAVIOR 
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STATINTL 


REQUIREMENTS vs SPECIFICATIONS 


REQUIREMENT 


PROVIDE STANDARD ARITHMETIC OPERATIONS 
FOR AT LEAST TWO YEARS 

IN A SHIRT-POCKET SIZE 

TO 8 DIGIT ACCURACY 


SPECIFICATION 
PERFORM ADDITION, SUBTRACTION, MULTIPLICATION & DIVISION 


MTBF = 3 YRS EXCEPT POWER; FIELD REPLACEABLE BATTERIES, 
MTTR = 20 MIN 


3x 5x 1/4 in. PACKAGE 
10 PLACE FIXED POINT DISPLAY, 12 DIGIT INTERNAL ACCURACY 
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IMPLEMENTATION EFFECTS 


CONFIDENCE LEVELS OF ATTAINABLE PERFORMANCE 
EASE OF IMPLEMENTING “SOFT”? REQUIREMENTS 
INTRODUCTION OF “DERIVED”? REQUIREMENTS 
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STATINTL 


DISCIPLINES USED TO DEVELOP REQUIREMENTS 


INQUIRY 
DESCRIPTION OF OPERATIONAL PROBLEM 
DEFINITION OF TERMS 
FACT FINDING OF USER NEEDS 
ANALYSIS 
SIMULATION 
SENSITIVITY ANALYSES 
PERFORMANCE LIMITS 
TRADE-OFF STUDIES 
COST-PERFORMANCE TRADES 


CAPABILITY vs DEVELOPMENT TIME 
RISK vs MERITS 
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STATINTL [ | 
DEFINING GOOD REQUIREMENTS (continued) 


DECOMPOSE 


WHAT ARE MY AIMS? 

WHAT ARE SOME MEASURABLE GOALS? 

WHAT OBJECTIVES MUST BE MET ALONG THE WAY TO THE GOALS? 
WHAT EXISTS? 

WHAT’S MISSING? 


IDENTIFY LIMITING FACTORS 


COST 
TECHNOLOGY 
INSTITUTIONAL 
PERSONNEL 
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TATINTL 
DEFINING GOOD REQUIREMENTS continuea) 


DEFINE THE RULES 


LEGAL 
POLICY 
TRADITION 


ADOPT DIFFERENT VIEWPOINTS 


USERS 
BUILDERS 
FUNDERS 


CONSIDER THE CONSEQUENCES? 


IMMEDIATE 
NEAR TERM 
FAR TERM 
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STATINTL [ 
PRINCIPLES OF STATEMENTS OF WORK 
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THE ESSENCE OF A SOW 


REQUIREMENTS STATE WHAT THE END PRODUCT MUST DO TO SATISFY THE USER 
THE SOW STATES WHAT THE CONTRACTOR MUST DO TO SATISFY THE CONTRACT 
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PRODUCT ORIENTED SOW 


THE PRODUCT MUST: 


MOVE 

SENSE 

COMPUTE 

PROTECT 

FUNCTION RELIABLY 

BE COMPATIBLE WITH A DEFINED OPERATIONAL ENVIRONMENT 
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PRODUCT ORIENTED SOW 


THE PRODUCT MUST: 


MOVE 

SENSE 

COMPUTE 

PROTECT 

FUNCTION RELIABLY 

BE COMPATIBLE WITH A DEFINED OPERATIONAL ENVIRONMENT 
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PRODUCT ORIENTED SOW  ntinued) 


THE CONTRACTOR MUST: 


DERIVE SPECIFICATIONS 

DOCUMENT DEVELOPMENT REQUIREMENTS 

DESIGN THE PRODUCT 

PRODUCE AND MAINTAIN DRAWINGS & PRODUCT SPECS 
HOLD DESIGN REVIEWS 

PROVIDE A PRODUCTION FACILITY 

BUILD THE PRODUCT 

MAINTAIN QUALITY CONTROL 

MAINTAIN CONFIGURATION CONTROL 

TEST THE PRODUCT : 

PRODUCE USER AND MAINTENANCE MANUALS 

PROVIDE TRAINING 

DELIVER AND INSTALL THE PRODUCT 

PRODUCE PERIODIC COST AND TECHNICAL PROGRESS REPORTS 
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STATINTL 


R&D ORIENTED SOW 


THE PRODUCT MUST: 


DEFINE LIMITS 
ESTABLISH COST-PERFORMANCE OPTIONS 
ESTABLISH FEASIBILITY 


THE CONTRACTOR MUST: 


PERFORM ANALYSES 
CONSTRUCT A WORKING MODEL 
CONDUCT EXPERIMENTS 
DOCUMENT RESULTS 

PROJECT FUTURE PERFORMANCE 
CONDUCT BRIEFINGS 
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STUDY ORIENTED SOW 


THE PRODUCT MUST: 


ILLUMINATE A PROBLEM 
DEFINE AND EVALUATE OPTIONS 
ANALYTICALLY ESTABLISH BEHAVIOR 


THE CONTRACTOR MUST: 


OBTAIN DATA 

DEVELOP MODELS 

CONDUCT ANALYSES 

EVALUATE RESULTS 

HOLD MEETINGS 

DOCUMENT RESULTS 

CONDUCT A MID-TERM & FINAL BRIEFING 
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THE COST CONNECTION OF THE SOW 


PRODUCT COSTS: 


FUNCTION 
PERFORMANCE 
QUANTITY 
QUALITY 


PROCESS COSTS: 
TASKS 


DESIGN 
TEST 
TRAINING 


DOCUMENTATION 


SPECS 

DRAWINGS 

PLANS & PROCEDURES 
FIN.” NCIAL REPORTS 
SCHEDULE PROJECTIONS 


EXTERNAL COORDINATION 


USER INTERFACES 
TECHNICAL INTERFACES 


INFORMATIVE MPNICLCIAIMG 
UNE VUAWVIA TIVO OMICET INGO 
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THE COST CONNECTION OF THE SOW continued) 


WORK BREAKDOWN STRUCTURE: 
PRODUCT CONNECTED TASKS: 


PRODUCT SPECS 
PRODUCTION 
UNIT TEST 


PROCESS MANDATED TASKS: 


SYSTEMS ENGINEERING 

QUALITY CONTROL 

SYSTEM TEST 

CONFIGURATION MANAGEMENT 
FINANCIAL & SCHEDULE CONTROLS 
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SOW CHECKLIST 


DEFINES ONLY THAT WHICH IS NECESSARY FOR THIS CONTRACT 
GENERAL INFORMATION IS SEPARATED FROM DIRECTION 
CONTRACTOR OBLIGATIONS ARE CLEAR AND DIRECT 
ELIMINATES EXTRANEOUS DATA REQUIREMENTS 

TELLS THE CONTRACTOR WHAT HE MUST DO, NOT HOW TO DO IT 


SUFFICIENTLY SPECIFIC TO ALLOW RESOURCES TO BE LISTED 
STIPULATES DELIVERABLES, QUANTITIES & DATES 


REFLECTS THE MANAGEMENT CONCEPT FOR EXECUTION OF THE CONTRACT 


CONSISTENT WITH CONTRACT DATA REQUIREMENTS LIST (CDRL) 
REFERENCED CITATIONS ARE CHECKED FOR PERTINENCE 


REQUIRES THE MINIMUM CHANGE TO THE CONTRACTOR’s NORMAL. PRACTICES 
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INTRO TO COST & SCHEDULE ANALYSIS 
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TATINTL 
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STAGES IN COST & SCHEDULE ANALYSIS 


IDENTIFY PRODUCT ELEMENTS 

DEFINE TASKS TO PRODUCE EACH 

DETERMINE COST ELEMENTS 

DERIVE COST ESTIMATION RELATIONSHIPS 

DETERMINE SCHEDULE CONNECTIONS & CONSTRAINTS 
MINIMIZE COST-RISK TRADE-OFF 
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COST ELEMENTS 


DIRECT COSTS 


LABOR 

EQUIPMENT 
MATERIALS 
PURCHASED ITEMS 
TRAVEL 

SHIPPING 
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COST ELEMENTS (continued) 


INDIRECT COSTS 


HOLIDAYS & VACATION PAY 

INSURANCE 

UTILITIES 

SECURITY 

PAYROLL & ACCOUNTING _ 

GENERAL PURPOSE RESEARCH FACILITIES ~ 
BID & PROPOSAL BUDGETS 

INDEPENDENT RESEARCH & DEVELOPMENT 


FEE 


EARNED VALUE 
DELIVERED PRODUCTS 
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[| 


TASK DEFINITIONS 


KEYED TO END PRODUCTS & DELIVERABLES 

SUPPORT INTERNAL MILESTONES 

HIERARCHICAL WORK STRUCTURE TO MEET MILESTONES 
DEFINES A WINDOW INTO THE SMALLEST UNIT OF PRODUCTION 
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SCHEDULE CONSTRAINTS 


RESOURCE LEVELS 

PRECEDENT TASK DEPENDENCY 
USE OF SHARED FACILITIES 
CALENDAR DRIVEN MILESTONES 
EXTERNALLY DEFINED INTERFACES 
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STATINTL 


SCHEDULING CONCEPTS 


MINIMUM TIME PATH 

MINIMUM COST PATH ? 
MAXIMUM (MINIMUM SLACK) PATH 
LOAD LEVELING 

EARLIEST vs LATEST START DATE 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


- Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


STATINTL 


SCHEDULING ALGORITHMS 


FORWARD SCHEDULING 
RESOURCE AVAILABILITY 
TASK DEPENDENCIES 
LOAD LEVELING 

BACKWARD SCHEDULING 


NEED DATES 
SLACK MANAGEMENT 
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TATINTL 


COST-SCHEDULE TRADES 


OVERTIME PAY 

OVERHEAD STRETCH-OUT 
LEARNING CURVE EFFECTS 
PRODUCTION LINE LIMITS 
FIXED FACILITY CHOKEPOINTS 
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STATINTL 


LIFECYCLE FACTORS 


MAINTENANCE 

SUPPLIES 

RECURRING TRAINING 
OPERATIONAL MODIFICATIONS 
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REQUIREMENTS 
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EVOLVING DETAIL 


NEEDS (USER/systems engineer) 
IDEAL GOALS 
MISSION ORIENTED 
TEST: MUST MAKE OPERATIONAL SENSE 
REQUIREMENTS (USER/SYSTEMS ENGINEER) 
PERFORMANCE GOALS 
MIX OF MISSION, ARCHITECTURE, AND FUNCTION 
TEST: MUST BE FEASIBLE 
SPECIFICATIONS (user/SYSTEMS ENGINEER) 
MEASURABLE PROPERTIES 


EQUIPMENT & FUNCTIONS 
TEST: MUST BE PROVABLE 
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STATINTL 


INTERFACE SPECIFICATIONS 


HARDWARE 


PACKAGING CONSTRAINTS 
PHYSICAL INTERFACES 
ELECTRICAL INTERFACES 
COMPUTERS 

CONSTRAINTS OR LIMITATIONS 


SOFTWARE 
OPERATING SYSTEM 
UTILITY PROGRAMS 


OTHER APPLICATION PROGRAMS 
CONSTRAINTS OR LIMITATIONS 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


INTERFACE SPECIFICATIONS. (continued) 


COMMUNICATIONS 


PROTOCOLS 
PRIORITIES 
TRAFFIC VOLUME 
TIMELINESS 


DATA 


INPUT/OUTPUT INFORMATION 
QUANTITY | 
TIMING 


OPERATIONS 
ORGANIZATIONAL ROLES 


PROCEDURES 
INTERACTIONS 
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DATA PROCESSING SYSTEMS 


FUNCTIONS AND OPERATING MODES 
INITIALIZATION 

WORKLOAD THROUGHPUT & PEAKS 
DATA CHARACTERISTICS & VOLUME 
STIMULI AND RESPONSES | 
STORAGE/RETRIEVAL VOLUMES & TIMELINES 
ACCURACY 

INPUT/OUTPUT OPTIONS AND LOADS 
DECISION AND CONTROL OPTIONS 
ERROR DETECTION & RECOVERY 
HUMAN FACTORS 

DESIGN MARGINS 

AVAILABILITY 
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SENSOR SYSTEMS 


~ SIGNATURE ENVIRONMENT 


BACKGROUND ENVIRONMENT 
DETECTION VOLUME 

PROBABILITY OF DETECTION 

FALSE ALARM RATE 

FAILURE DETECTION & RECOVERY 

BUILT-IN TEST 

AVAILABILITY 

OPERATING LIFE 

PACKAGING LIMITS 

SUPPORT LIMITS (POWER, TEMP CONTROL, 
ETC) | 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160004-9 


TATINTL 


MECHANICAL SYSTEMS 


EXTERNAL LOADS 
_ FUNCTIONS 

ACCURACY/TOLERANCES 

SAFETY 

WEAR LIMITS 

MOTION CONSTRAINTS 

OPERATING ENVIRONMENTS 

OPERATING LIFE 

AVAILABILITY 

MEAN-TIME-BEFORE-FAILURE & MEAN-TIME-TO- 

REPAIR 
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NEGATIVE REQUIREMENTS 


IF | ONLY STATE THAT WHICH | WANT 

AND NEVER SAY WHAT | WANT NOT 

THEN | CAN’T COMPLAIN WHEN | FINALLY LEARN 
THAT TOO MUCH ....NOT.... 

IS ALL | GOT! 
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PERFORMANCE vs “...ILITIES” 


ENVIRONMENTAL DESCRIPTION 
BACKGROUND ENVIRONMENT 
EXTERNAL LOADS 
OPERATING ENVIRONMENTS 

PERFORMANCE 
MISSION UTILITY 


ERROR ANALYSIS 
ACCURACY/TOLERANCES 
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TATINTL 


VARYING LEVELS OF PRESUMED THREAT LEVELS 


TIMELINESS 


SYSTEM SIMULATION UNDER LOAD 
MAN/MACHINE TRADE-OFFS 
COST/COMPUTER POWER TRADE-OFFS 
REACTION TIME 


ROBUSTNESS 
PHYSICAL SURVIVABILITY 
ADAPTABILITY 
TRANSPORTABILITY 
CONTROL OPTIONS 
ACTIVITY LEVELS 


MISSION AND OPERATIONS ANALYSES 
ENVIRONMENTAL DESCRIPTION 
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MANNED vs UNMANNED SYSTEMS 


DECISION ANALYSIS 
DECISION SEQUENCES 
INFORMATION REQUIREMENTS 
ERROR EFFECTS 
WORK LOAD ANALYSIS 
CONTENT 
LEVEL 
INTERFACES 
HUMAN FACTORS STUDIES 
PHYSICAL RESPONSE 


STRESS RESPONSE _ 
TRAINING 
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STATINTL 


FACTORY vs FIELD CALIBRATION © 


RELIABILITY 

MAINTAINABILITY 

ERROR DETECTION & RECOVERY 
SURVIVABILITY 
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CONTRACTOR vs GOVERNMENT MAINTENANCE 


SURVIVABILITY 
LIFECYCLE COSTS 
FOREIGN OPERATIONS 
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SCOPE OF DEFINITION 


Transition from Current Operations 
Operational Organizational Roles 
Support Organization Roles 
Training 

Maintenance 

End-of-Life Disposal 


/ 
/ 
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OPERATIONAL DYNAMICS 


EXTERNAL INTERFACE DYNAMICS 
PROCESS FLOWS 

CYCLES 

SEQUENCES 

TIMELINES 
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OPERATIONAL THEMES 


DATA ENVIRONMENT 
THREAT ENVIRONMENTS 
LOGISTIC ENVIRONMENTS 
PERSONNEL 

CONTROL CRITERIA 
PERFORMANCE LEVELS 
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TRANSITION FROM CURRENT OPERATIONS 


OPERATIONAL CONTINUITY 
FALLBACK MODES 
POST-TRANSITION EVALUATION 
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USERS (‘“‘BENEFACTOR’”’) 


ROLES 


DEFINE PROBLEMS 
DEVELOP STRATEGY 
STRUCTURE APPROACH 
MAKE DECISIONS 
EVALUATE RESULTS 


CONSTRAINTS ON THE ROLE 
SPAN OF CONTROL IN OPERATIONS 
UNDERSTANDING OF THE SYSTEM 
INFORMATION SOURCES & CONTENT 
REQUIREMENTS IMPACT 
PERFORMANCE ASSESSMENT 
OPERATIONAL FLEXIBILITY 
USER-ORIENTED TRAINING 
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OPERATORS (PHYSICAL USER”) 


ROLES 


ENTER DATA 

CALIBRATE EQUIPMENT 
PERFORM TEST PROCEDURES 
MANIPULATE DEVICES 
VALIDATE RESULTS 

RESPOND TO ERRORS 


CONSTRAINTS ON THE ROLE 


EXPERIENCE 

EDUCATION LEVEL 
PHYSICAL CAPABILITIES 
OPERATING ENVIRONMENT 


REQUIREMENTS IMPACT 


DECISION OR REACTION TIME 
ERROR EFFECTS ON MISSION 


SAFETY & SECURITY RISKS 
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SUPPORT ORGANIZATION ROLES 


COOPERATIVE MISSION FUNCTIONS 
COLLATERAL DATA SOURCES 
SERVICES | 
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TRAINING 


MODES 
INITIAL 
RECURRING 
QUICK REACTION 
AUDIENCES 
USERS 


OPERATORS 
MAINTENANCE 
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TATINTL 
MAINTENANCE 


PREVENTATIVE 
ON-DEMAND 
REPLACEMENT PARTS 
CONSUMABLE SUPPLIES 
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The Operational Environment 


EVENTS ae ie RELATIONSHIPS 
/ 
/ 


ALL 
PROBLEMS 


SYSTEMS 
SYSTEMS 


OPERATORS 


POLICIES 
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ACQUISITION PLANNING & QUALIFYING SOURCES 
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ACQUISITION PARAMETERS AND THEIR INFLUENCE 


SIZE 
NUMBER OF CONTRACTS 
MANAGEMENT STRUCTURE 
RISK 
SCHEDULE 
BUDGET 


TYPE OF CONTRACT 
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STATINTL 


CONTRACTING STRUCTURE 


COMPETITIVE vs SOLE SOURCE 

MULTIPLE CONTRACTS 

DIVIDING A JOB 

AUXILIARY SUPPORT CONTRACTS 
SYSTEM INTEGRATION 
INDEPENDENT TEST 
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CONTRACTUAL PACKAGING 


DESIGN FACTORS 


INTERFACE SIMPLICITY 
TESTABILITY 


MANAGEMENT FACTORS 


INDUSTRY SPECIALIZATION 
SCHEDULE 
ACCOUNTABILITY 
INTERDEPENDENCES 
SECURITY 
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AWARD PHASING 


PRODUCT DEPENDENCIES 

BUDGETARY REQUIREMENTS 

TECHNICAL DATA EXCHANGE 
GOVERNMENT SUPPORT REQUIREMENTS 
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PEAK GOVERNMENT SUPPORT REQUIREMENTS 


REQUIREMENTS 
STATEMENT OF WORK 
REQUEST FOR PROPOSAL 
PROPOSAL EVALUATION 
SPECIFICATION REVIEW 
DESIGN REVIEW 

TESTING 
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STATINTL i 
PROCUREMENT 


Market Survey 
Request for Proposals 
Proposal Evaluation 
‘Contracting 
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STATINTL fl | | 
MARKET SURVEYS 


Critical Skills 

Security & Foreign Ownership/Control 
Financial Stability 

Specialized Facilities 

Manpower | 

Past Performance (CONIF) 
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TATINTL CF 
REQUEST FOR PROPOSALS 


STATEMENT OF WORK 


TASKS 
DELIVERABLES 
SCHEDULE 

SYSTEM DESCRIPTION 


PRODUCT REQUIREMENTS 

CONTRACT DOCUMENT APPROVAL 

DATA REQUIREMENTS DOCUMENT 

WORK BREAKDOWN STRUCTURE © 
INCENTIVE OPTIONS 

PROPOSAL ORGANIZATION REQUIREMENTS 
PROPOSAL EVALUATION GUIDES 
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TECHNICAL PROPOSAL CONTENTS 


DESIGN CONCEPT 


HARDWARE 

SOFTWARE 

DATA 

OPERATIONAL PROCEDURES 


TRAINING 
MAINTENANCE 


CRITICAL PROBLEMS 


DIFFICULT REQUIREMENTS 
IMMATURE TECHNOLOGY 
STRESSFUL ENVIRONMENT 
UNUSUAL LOGISTIC NEEDS 
PACKAGING 
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TECHNICAL PROPOSAL CONTENTS (continued) 


TRADE-OFF OPPORTUNITIES 


CONFLICTING REQUIREMENTS 
COST-EFFECTIVE ENHANCEMENTS 
DEVELOPMENT COST vs RECURRING COSTS 


DEVELOPMENT APPROACH 


PROTOTYPES 
BUILDS 
PHASED DEPLOYMENTS 


TEST APPROACH 


TEST PHILOSOPHY 
GOVERNMENT ROLE & DATA 
SIMULATORS 

TEST EQUIPMENT 


SCHEDULE 
MANNING 
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STATINTL | | 
MANAGEMENT PROPOSAL CONTENTS 


Organization and Personnel 
Technical and Financial Controls 
Configuration and Data Management 
Security Plan 

Developement Support 

Field Support 

Integrated Logistics Support 
Documentation Plan | 
Relevant Experience 
Biographical Data 

Proposal Linkage 
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TATINTL 


COST PROPOSAL CONTENTS 


ESTIMATING BASIS 
TECHNICAL SIZING 
PRODUCTION BASE 
EXISTING FACILITIES, TOOLING, ETC 


WORK BREAKDOWN STRUCTURE 
RATES 


DIRECT LABOR 
OVERHEAD 


RESULTS 
PRODUCT RELATED BREAKOUT 
FISCAL YEAR SPREAD 
REQUIREMENTS BREAKOUT 
EFFECTS OF UNCERTAINTIES 
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TATINTL 


PROPOSAL EVALUATION GUIDES 


FORCE EARLY RECOGNITION OF CRITICAL ISSUES 


KEY DESIGN TRADE-OFFS 
STATE-OF-ART UNCERTAINTIES 
RAPID MANNING REQUIRED 
FAMILY SEPARATIONS 
UNCONTROLLED INTERFACES 


STIMULATE THE DESIRED BALANCE 
TECHNICAL/MANAGEMENT/COST 
DESIGN/TEST/OPERATIONS 
ARCHITECTURE-SPECIFIC ELEMENTS 

PROMOTE COMPETITION 
ENCOURAGE RESPONSES 


INCREASE ACCEPTABLE RESPONSES = 
IMPROVE CONTRACTOR'S UNDERSTANDING OF PRIORITIES 


’ 
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ESTIMATING COST 
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TATINTL —— 7 
COST ESTIMATION 


TOTAL EFFORT 


PROJECT SCALE 
COMPLEXITY 

AVAILABLE TECHNOLOGY 
DELIVERY TIME 


LOADING 


CUMULATIVE EFFORT 
STAFF ABSORBTION RATE 
WORK DAY LENGTH 
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STATINTL fed 
COST ESTIMATING METHODS 


DETAILED ESTIMATES 
SCALED ANALOGS 
PARAMETRIC MODELS 
SUBJECTIVE METHODS 
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DETAILED ESTIMATES 


DESCRIPTION _ 


BASELINE DESCRIPTION OF WORK 

CAPITAL EQUIPMENT & SCHEDULES 
COSTING RATES AND FACTORS 
WORK-BREAKDOWN-STRUCTURE ORIENTED 
“LEARNING CURVES” 


PROS/CONS 
HIGHLY VISIBLE 


GAINS COMMITMENT 
COSTLY ; 
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STATINTL 


SCALED ANALOGS 


DESCRIPTION 
EXISTING SIMILAR SYSTEM 
CAN BE SCALED PHYSICALLY OR 
PROGRAMMATICALLY 
PROS/CONS 


HIGH CONFIDENCE BASELINE 
LIMITED EXTRAPOLATION 
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| 


TATINTL 
PARAMETRIC MODELS 


DESCRIPTION 


BASED ON EMPERICAL FIT TO HISTORICAL DATA 
COST ELEMENT ORIENTED 
COMPONENT BREAKDOWN 
TECHNICAL, PHYSICAL OR 
PERFORMANCE CHARACTERISTICS 
PROGRAM TASKS (QUALITY CONTROL, 
DOCUMENTATION, DESIGN) 


PROS/CONS 


MODELS PERMIT PARAMETRIC DESIGN 
REPEATABLE 

INTERCONNECTED EFFECTS 

SCHEDULE SENSITIVE | | 

DATA BASES MUST BE VALIDATED 

MODELS MUST BE CALIBRATED 
EXTRAPOLATION LIMITS SHOULD BE KNOWN 
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TATINTL | 
SUBJECTIVE METHODS 


DESCRIPTION 
EXPERT OPINION 
DELPHI 
W-A-G 
PROS/CONS 
ONLY TECHNIQUE USEFUL FOR NEW TECHNOLOGY 


FAST & CHEAP 
UNBOUNDED ERRORS 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


TATINTL 


ESTABLISHING SCALING FACTORS 


i 


EFFECTS OF SIZE 
TECHNOLOGY IMPROVEMENTS 
PRODUCTION TOOLS 
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FINDING THE DATA 


INDUSTRY MODELS 
CONTRACTED STUDIES 
GOVERNMENT FILES 
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TATINTL 
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_ 


LOOKING BEHIND THE DATA 


SHARED FACILITIES 
GOVERNMENT FURNISHED ITEMS 
PRODUCTIVITY OR EXPERIENCE 
REQUIREMENTS CHANGES 
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SCHEDULING FACTORS 


STAFFING RATE POSSIBLE 
LEARNING ACCELERATION FACTOR 
INTERFACE COMPLEXITY | 
RESOURCE BOTTLENECKS 
CONTRACT DEPENDENCIES 
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Work Breakdown Structure 


tions 
Development Test Production Opera 


Systems 


Management 


Support 


Engineering 


General Mgt. Interface Definition Equipment Tooling Training 


Costs & Schedule Requirements Design Test Materials Production Spare Parts 
Control Analysis Prototype Test Conduct Quality Control Documentation 
Document System Architecture Production Model Analysis/ 
Maintenance Reporting 
Performance Unit Test 
Configuration Monitoring 
17 Control Executive 
Design j 
Code henieeenetenheate | 
Walkthrough r 


Unit Test 


Integration 
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Shortfall 


interfaces 


TIME —> 


a 


Excessive 
Tu mover 
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DEFINES SCOPE OF EFFORT 


MANAGEMENT 
STUDIES 
DEVELOPMENT 
TESTING 
COORDINATION 
INTEGRATION 
MEETINGS 
TRAINING 
REVIEWS 
MAINTENANCE 
SUPPORT 
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TASK EXAMPLES 


DEVELOP, TEST, AND DELIVER A SYSTEM WHICH MEETS 
REQUIREMENTS SPEC 

PROVIDE DOCUMENTATION PER MILSTD 490/483 

PERFORM AN ACCURACY/RELIABILITY TRADEOFF STUDY 

TRAIN 200 PERSONNEL AT 3 CONUS SITES 

SHARE IN DEVELOPMENT OF INTERFACE SPECS 

REVIEW & CRITIQUE OTHER CONTRACTOR’S TEST PLANS 

PROVIDE A MONTHLY PROGRESS BRIEFING IN WASHINGTON 
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STIPULATES REQUIRED MANAGEMENT PROCEDURES 


CONFIGURATION CONTROL 
QUALITY CONTROL 

CRITICAL PATH SCHEDULES 
WORK BREAKDOWN STRUCTURE 
COST REPORTING SYSTEM 
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STATINTL 


IDENTIFIES OTHER APPLICABLE DOCUMENTS 


STANDARDS AND PRACTICES 
REGULATIONS 

LEGALLY REQUIRED PROGRAMS 
PRECEDENCE OF REQUIREMENTS 

HIGHER LEVEL REQUIREMENT DOCUMENTS 
INTERFACE DOCUMENTATION 
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IDENTIFIES DELIVERABLES AND SCHEDULE 


DOCUMENTS 
BRIEFINGS 

DESIGN REVIEWS 
STATUS MEETINGS 
END PRODUCTS 
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SPECIFIES ENVIRONMENT (where appropriate) 


FACILITIES 

OPERATING SYSTEM 

OTHER PROGRAMS 

GOVERNMENT FURNISHED RESOURCES 
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ESTABLISHES THE NEED FOR 
MINIMAL STANDARDS 


Programming Standards and 
Conventions 


Design Standards 

Testing and Quality Assurance 
Safety Criteria | 
Physical & Personnel Security 
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GIVES SUPPLEMENTAL INFORMATION 


SYSTEM DESCRIPTION 

OPERATIONAL CONCEPTS — 

SYSTEM DATA FLOW 
INTER-CONTRACTOR DATA EXCHANGES 
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WORK WORD—EXAMPLES 


ANALYZE—Solve by analysis 
CONSIDER—Think about 

DEVELOP—Bring into being 

DEFINE—Make clear; Settle the limits 
ESTIMATE—Approximate an opinion of 
INSTALL—Place; Put into position 
INSPECT—Examine carefully or officially 
MANUFACTURE—Fabricate from raw materials 
ORGANIZE—Integrate; Arrange in a coherent unit 
PERFORM—Do; Carry out; Accomplish 
RECOMMEND—Advise; Attract favor of 
STUDY—Careful examination or analysis 
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CONTRACT DOCUMENTATION 
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CONTRACT DATA REQUIREMENTS LIST 


DATA ITEM TITLE AND CRDL NUMBER 

DATA ITEM DESCRIPTION & MODIFICATIONS 
REFERENCE SOW PARAGRAPH 

SCHEDULE & DISTRIBUTION 

APPROVAL 
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STATINTL 


CONTRACT DOCUMENT APPROVAL 


PROVIDES BASELINE FOR DESIGN 
ENABLING AUTHORITY FOR INCENTIVE EVENTS 
FACILITATES ULTIMATE QUALITY CONTROL 


REVIEW CYCLE MUST BE REALISTIC 
REVIEWERS MUST BE CRITICAL 


PERMITS COORDINATION ACROSS CONTRACT INTERFACES 


USER COMMUNITY REVIEW 
TECHNICAL INTERFACES 


ASSURES THAT CONTRACTOR PRODUCES INTERNALLY 
REQUIRED DOCUMENTATION 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


TATINTL 
DOCUMENTATION PURPOSES 


STRUCTURE THE IMPLEMENTATION 


ARCHITECTURE 
LOGIC 
DATA STRUCTURE 
USAGE 


GUIDE THE DEVELOPMENT PROJECT 


PLANS 

STATUS 

TESTS 
COORDINATION 


FACILITATE OPERATIONAL USAGE 


INPUTS 
STATUS 
CHANGES 
STUDIES 
TRAINING 
MAINTENANCE 
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; sh Hee Nile 


~ The Software Document Maze 


-—________-_—- DEVELOPMENT SEQUENCE 


PROJECT SOFTWARE PRELIMINARY CRITICAL END- 
STARTUP REQUIREMENTS DESIGN DESIGN PRODUCT 
i REVIEW REVIEW REVIEW ACCEPTANCE 
APPROVAL OF J 
SYSTEM 


TOP LEVEL 
PLAN REQUIREMENTS | system — | SOFTWARE SOFTWARE 


DETAILED 


DESIGN SYSTEM SOFTWARE 
INTEGRATE DT&E TEST TEST 
REQUIREMENTS DEVELOPMENT | PRODUCT DELIVERED INTEGRATED 


SPECIFICATION SPECIFICATION | SPECIFICATION PRODUCT TEST SYSTEM TEST 


TYPEA B-5 C-5 CPCI TEST PLAN SYSTEM TEST PLAN 


SEGMENT PARTI 


PART Il SEGMENT TEST PLAN | DEMONSTRATION PLAN 


END PRODUCT 
ACCEPTANCE PLAN 


S/W REQUIREMENTS 


SUBSYSTEM PROGRAM OT&E PLAN 
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PROJECT CONTROL DOCUMENTS 


MANAGEMENT PLANS 


ORGANIZATION 

STAFFING 

FINANCIAL AND SCHEDULE CONTROLS 
SUPPORT RESOURCES 

SECURITY 


QUALITY ASSURANCE 


CONFIGURATION MANAGEMENT PLAN 

DESIGN STANDARDS AND CONVENTIONS 

COORDINATE SYSTEM DEFINITIONS 

PROGRAMMING PRACTICES, 
STANDARDS AND CONVENTIONS 

TEST PROGRAM AND TEST DATA PLAN 
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TATINTL 


PROJECT CONTROL DOCUMENTS continued) 


MANAGEMENT STATUS REPORTS 


FINANCIAL DATA 
CRITICAL PATH ANALYSES 
TECHNICAL PROBLEMS 
COMPUTER UTILIZATION 


SUPPORT SYSTEMS 
DOCUMENTATION 
TRAINING 


SECURITY 
ACTIVATION 
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TATINTL se 
THE DATA ITEM DESCRIPTION 


TITLE, NUMBER, DATE, AND OFFICE OF 
PRIMARY RESPONSIBILITY 

USE (PURPOSE WITHIN OR FOLLOWING 
THE CONTRACT) | 

INTERRELATIONSHIP (PRECEDENT OR DE- 
PENDENT DOCUMENTS) 

REFERENCES 

PREPARATION INFORMATION 


TABLE OF CONTENTS 
APPLICABLE DOCUMENTS 
BOILERPLATE 

DETAILED CONTENT 


NON-REPETITIVE 
REPETITIVE 


FIGURES 
GLOSSARY 
INDEX 
APPENDICES 
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PROPOSAL EVALUATION 


OBJECTIVES 
PROCESSES AND PROCEDURES 


LEGAL FACTORS 
MANAGEMENT FACTORS 


CRITERIA 


ELEMENTS 
SCORING 
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STATINTL 
PROPOSAL EVALUATION OBJECTIVES 


ELIMINATE UNRESPONSIVE OR INFEASIBLE APPROACHES 
SOME BIDDERS DON’T KNOW THEY LACK NECESSARY 
BACKGROUND 
STATE OF THE ART IS NOT UNIFORMLY DISTRIBUTED 
OBTAIN BEST VALUE FOR THE GOVERNMENT 
SHAVE UNNECESSARY FRILLS 
ASSURE EFFORT COMMENSURATE WITH APPROACH 
TRADE-OFF COST AND PERFORMANCE 
ENFORCES OBJECTIVITY 
MINIMIZE PERSONAL INFLUENCES 


MAXIMIZE DEPENDENCY ON VERIFIABLE, RELEVANT, 
AND BINDING INFORMATION 
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TATINTL 


| | Proposal Evaluation Process 


PROPOSAL 


EVALUATION CONTRACT 
TEAMS FILES 


NEGOTIATION 


SOURCE ICT 


SELECTION 
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WEIGHTED 
CRITERIA , 
EVALUATION 
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PROPOSAL EVALUATION PROCEDURES 


ELIMINATING BIASES 


ROTATING ORDER 
NORMALIZING SCORES 
LIMITING EARLY INTERACTIONS 


KEEPING RECORDS 


PARTICIPANTS 

CRITERIA - ; 
WRITTEN CRITIQUES 
DISQUALIFYING FACTORS 
MINUTES OF DISCUSSIONS 
VOTING 


USE OF SPECIALISTS AND CONSULTANTS 


PROBLEM PECULIAR 
PROPOSAL PECULIAR 
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TYPICAL DEVELOPMENT PROBLEMS 


HARDWARE 


ACCURACY, SPEED, SENSITIVITY 
PACKAGING DENSITY 
WEIGHT/VOLUME/POWER 
RELIABILITY 

PHYSICAL SURVIVABILITY 
PRODUCTION QUALITY 


SOFTWARE 


COST & SCHEDULE ESTIMATION & CONTROL 

IN-LINE DEPENDENCE ON OPERATING SYSTEM & COMPILER 
HIDDEN ERRORS 

DOCUMENTATION 

MAINTENANCE COSTS 

TRAINING REQUIREMENTS 

REQUIREMENTS GROWTH 

UNDERESTIMATED COMPUTE-POWER 
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TECHNICAL CRITERIA 


REQUIREMENTS AUDIT 


TRACEABILITY 
IMPLEMENTATION 


FEASIBILITY 


PRIOR APPLICATION 

DETAILED SIZING AND TIMING STUDIES 
ERROR ANALYSES 

SENSITIVITY OR TRADE-OFF STUDIES 
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TATINTL 


TECHNICAL CRITERIA Continued) 


COMPLETENESS 


SOFTWARE 


DATA BASE ARCHITECTURE 
MATHEMATICAL ALGORITHMS 
HUMAN INTERFACES 

CONTROL OPTIONS 
OPERATING SYSTEM FEATURES 
DEVELOPMENT TOOLS 


HARDWARE 


OVERKILL 


LOGIC 

MOTION ANALYSIS 
INTERFERENCE ANALYSIS © 
PRODUCABILITY 

RELIABILITY 

PACKAGING 

AUTOMATIC TEST EQUIPMENT 
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MANAGEMENT CRITERIA 


PEOPLE AND ORGANIZATION 


RELATIONSHIP TO PROPOSAL DEVELOPMENT 
TIME COMMITMENT AND CURRENT ASSIGNMENTS 
STAFFING RATE FEASIBILITY 

INDEPENDENT TEST AND INTEGRATION 


SYSTEMS AND PRACTICES 
EFFECTIVENESS 


TIMELINESS 
PROOF OF USE 
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STATINTL 


MANAGEMENT CRITERIA (continued) 


EXPERIENCE 


PRIOR SIMILAR JOBS 
LINKAGE TO INDIVIDUALS 
REFERENCES 


_ SCHEDULE 


INTERNAL MILESTONES 
REVIEW CYCLES 

CRITICAL PATH DEVELOPMENT 
CREDIBILITY 
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TATINTL | | 
COST CRITERIA 


BASIS. 


ENGINEERING ESTIMATES 
TOTAL SCOPE OF EFFORT 
REQUIRED LABOR MIX 
OVERHEAD ELEMENTS 


DEFINITIONS 
MAN-HOUR 


MONTH 
COMPUTER TIME 
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COST CRITERIA (continued) 


STANDARDS 


JUDGING CREDIBILITY 
ESTABLISHING ALLOWABLES 


GOVERNMENT RESOURCES 
COMPUTER TIME 
MAN POWER 
TRAVEL BUDGETS 

LIFECYCLE ESTIMATES 
LIFE-SPAN 


ACTIVITIES 
DISCOUNT RATE 
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— m7 | Proposal Evaluation Criteria 


@ TECHNICAL 
e REQUIREMENTS AUDIT 
e FEASIBILITY | RELEVANCY 
e COMPLETENESS 
e OVERKILL 


@ MANAGEMENT 

PEQPLE AND ORGANIZATION 
SYSTEMS AND PRACTICES 
EXPERIENCE 

SCHEDULE 


e COSTS 
BASIS SUBSTANTIATION 
DEFINITIONS 

STANDARDS 
GOVERNMENT RESOURCES 
LIFE CYCLE ESTIMATES 
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TATINTL 


Separating Fact From Fiction 


@ RELEVANCY 
e DOES IT HELP MEET THIS SYSTEMS OBJECTIVES? 
e DIDIT INVOLVE THESE PEOPLE? 
e HAS IT BEEN ASKED FOR? 
e WAS THAT ENVIRONMENT (noise, data rate, schedule, cost) 
AS CONSTRAINING? 


@ SUBSTANTIATION 
e IS IT ASSERTED OR PROVEN? | 
@ WHAT DATA HAS BEEN USED? WHAT IS ITS SOURCE? 
HOW WAS IT COLLECTED? 
® WERE RECORDS KEPT, OR IS THIS A RECONSTRUCTION? 
e WOULD A DIFFERENT GROUP OF PEOPLE OR SET OF DATA 
YIELD THE SAME RESULTS? 
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STATINTL 


STAFFING 


THE SURPRISE OF WINNING 
PROPOSAL TEAM TRANSITION 
INTERNAL PRIORITY. 
CLEARANCES 

PROJECT “REPUTATION” 
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TATINTL | | 
ORGANIZING 


CONFLICTING CONTRACTS 
POWER STRUGGLES 
DIFFERENCES IN APPROACH 
PERSONAL STRENGTHS 
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TATINTL 


INTERPRETATION OF REQUIREMENTS 


LIMITED MISSION OR TECHNOLOGY EXPERIENCE 
LIMITED PROJECT BACKGROUND 

MISDIRECTED PRIOR EXPERIENCE 
CONTRADICTORY OPERATIONAL INPUT 
AMBIGUOUS SPECIFICATION 
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TATINTL ae 
LONG LEAD TIME PURCHASES 


FAULTY PROJECT PLAN 
LOW CORPORATE PRIORITY 
INATTENTION 
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TATINTL 
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AVOIDING TROUBLE 


MEET ALL KEY STAFF 

TEST CONTRACTOR’S INTERPRETATION OF RQMTS 
CALIBRATE MORALE 

RECHECK VALIDITY OF PROJECT PLAN 

TEST IMPLEMENTATION OF PROJECT PLAN 

KEEP A TOTAL PROJECT PERSPECTIVE 

TRACK INITIAL STAFFING & ORGANIZATION 

WALK AND TALK 

MEET FREQUENTLY WITH PROJECT MANAGER 
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BE A CUSTOMER, NOT A GUEST 


ESTABLISH A POSITION OF BEING IN CONTROL 


INITIAL CONTACTS 

RFP 

BIDDERS CONFERENCE 
FACT-FINDING 
NEGOTIATIONS 


USE FORMALITY TO MAINTAIN CONTROL 
AGENDAS | 
MINUTES | 
ACTION ITEMS 
CONFIGURATION CONTROL 
USE INFORMALITY TO MAINTAIN RELATIONSHIPS 
PEOPLE ARE YOUR MOST IMPORTANT RESOURCE 
MANY REAL PROBLEMS REMAIN HIDDEN TO “THE SYSTEM” 
YOU PLAY A KEY ROLE IN MOTIVATING CONTRACTOR EMPLOYEES 
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TATINTL 


GOLDEN RULES OF ACQUISITION 


BE SPECIFIC 

BE CONSISTENT 
BE SKEPTICAL 
BE PERSISTENT 
BE TIMELY 
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TATINTL [ | 
TECHNICAL PROGRESS ASSESSMENTS 


STATUS MEETINGS 
TECHNICAL INTERCHANGE MEETINGS 
WORKING GROUPS 

PROGRESS BRIEFINGS 

SPECIAL AUDIT TEAMS 

DESIGN REVIEWS 
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TATINTL 


DESIGN REVIEWS IN DETAIL 


OBJECTIVES 
DOCUMENTATION 
PROCESS 

USEFUL PROCEDURES 
REVIEW MANAGEMENT 
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TATINTL 


DESIGN REVIEW OBJECTIVES 


MEASURE PROGRESS 


REFOCUS ATTENTION ON REQUIREMENTS, 
INTERFACES & OPERATIONS 


ASSURE COMPLIANCE 
UNCOVER OVERSIGHTS 
IDENTIFY COST-DRIVERS 
ELIMINATE UNNECESSARY FRILLS - 

PROVIDE DEFINITIVE INTERMEDIATE MILESTONES 
IMPROVES SCHEDULE ADHERENCE 
PERMITS NOVEL INCENTIVE STRUCTURE 
BASIS FOR CONFIGURATION CONTROL 

STIMULATE CONTRACTOR INTERNAL COMMUNICATIONS 
IMPROVED QUALITY | 
EARLY IDENTIFICATION OF PROBLEMS 


CONTRIBUTE TO TRAINING 
INTEGRATING CONTRACTOR 
OPERATORS/USERS 
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TATINTL 
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REFERENCE DOCUMENTS 


SYSTEM SPECIFICATION 
DEVELOPMENT SPECIFICATIONS 
PRODUCT SPECIFICATIONS 
CRITICAL PARTS LIST 

DATA DICTIONARY 

HDW/SW LIMITATIONS AND CONSTRAINTS 
INTERFACE CONTROL DOCUMENTS 
TEST PLAN 

TEST PROCEDURES 

TEST REPORT 

USERS MANUAL 

PROGRAMMING MANUAL 
MAINTENANCE MANUAL 
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TATINTL 


CONSTRUCTIVE DOCUMENT REVIEWS 


LOOK FOR THE BIG ISSUES 
REQUIREMENTS SATISFACTION 
COMPLETENESS | 
CONSISTENCY 
ACCURACY/CORRECTNESS 
KEEP EDITORIAL COMMENTS SEPARATE 
SAY WHAT YOU WANT & WANT WHAT YOU SAY 
AVOID PHILOSOPHIZING 
SUGGEST CORRECTION IF KNOWN 
AVOID SARCASM 
REMEMBER YOUR OBJECTIVES 
REQUIREMENTS 
SCHEDULE 
MONEY 
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TATINTL 


USEFUL PROCEDURES 


SYSTEM LEVEL REVIEWS 


' FORCES CONTRACTOR TO VIEW SYSTEM AS A WHOLE 


SAME PEOPLE/SAME ROOM/SAME TIME IMPROVES 
COMMUNICATION 


RESOLVES UNCERTAIN RESPONSIBILITIES 
COORDINATES RESPONSE TO CRITICAL PROBLEMS 


REQUIREMENTS TRACES 


DESIGN 
INTERFACES 
TEST PLAN 


DATA OR SIGNAL TRACES 


UNCOVERS INTERFACE PROBLEMS 
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REVIEW MANAGEMENT 


TEAM PREPARATION 


SELECTION 
TRAINING 
WORK LOAD BALANCE 


SETTING OBJECTIVES 


REQUIREMENTS PRIORITIES 
KEY RISKS/ISSUES 
SELECTED THEMES 


LOGISTICS 


DOCUMENT DISTRIBUTION 
COMMENT FORMS 
COMMENT COLLECTION 
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TATINTL 


REVIEW MANAGEMENT (continued) 


INTERNALLY COORDINATED RESPONSE 


ELIMINATES REDUNDANCY AND NOISE 
FOCUSES ATTENTION ON CRITICAL ITEMS 
DISCUSSION UNFOLDS NEW ISSUES 
DOCUMENTS INPUTS 


DOCUMENTED PROCEEDINGS 


RECORDS RATIONALE 
REDUCES MISUNDERSTANDINGS 
PROMOTES FULLY DEFINED ACTION ITEMS 
(who, what, when) 
SIGNATURE ESTABLISHES CUSTOMER’S CONTROL 


ACTION ITEM CLOSURE 


FOLLOW-UP REVIEW 
ACCOUNTING 
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ELEMENTS OF PROJECT CONTROL 


DESCRIBE THE SYSTEM IN MEASURABLE TERMS 


RACKS OF EQUIPMENT 
CIRCUIT BOARDS 
COMMERCIAL 
CUSTOM 
SOLDIER JOINTS 


OPERATOR STATIONS 
INPUT DEVICES 
CONVENTIONAL 
STATE OF THE ART 
DISPLAY OPTIONS 


SOFTWARE MODULES | 
NUMBER OF EQUATIONS OR ALGORITHMS 
TEXT BOOK | 
NEW DERIVATIONS 
LINES OF CODE 
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TATINTL 


ELEMENTS OF PROJECT CONTROL 


IDENTIFY DISCRETE MILESTONES ASSOCIATED WITH THE DESIGN, 
DEVELOPMENT, PRODUCTION, TEST AND SUPPORT OF THE IDENTIFIED 
SYSTEM 


PURPOSE DOCUMENTED 
FUNCTIONS DESCRIBED 
ARCHITECTURE APPROVED 
DESIGN COMPLETED 

FIRST TEST ATTEMPTED 
TEST COMPLETED 
INTEGRATION COMPLETED 


PLAN, MEASURE AND CONTROL THE EFFICIENT APPLICATION OF 
RESOURCES (PEOPLE, MATERIAL, RESOURCES, TIME) TO THE 
ACHIEVEMENT OF MILESTONES 


LEARNING/PREPARATION 
CREATING/USING 


VALIDATING/CORRECTING 
IDLE 


oF te be 
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DESIGN CONCEPT REVIEW OBJECTIVES 


ENHANCE AND CONFIRM UNDERSTANDING OF REQUIREMENTS 
IDENTIFY AND/OR MODIFY HIGH COST OR HIGH RISK REQUIREMENTS 
IDENTIFY, NEGOTIATE AND FREEZE EXTERNAL INTERFACES 

REVIEW AND VALIDATE CONTRACTOR’S DEVELOPMENT PLAN 
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DCR AGENDA 


OPERATIONS CONCEPT 

REQUIREMENTS 

TRADE STUDIES 

DEVELOPMENT PLAN — 

QUALITY ASSURANCE 

CONFIGURATION CONTROL 

REQUIREMENTS VERIFICATION 

PRELIMINARY DEVELOPMENT SPECIFICATIONS 
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TATINTL i = eaEREE ES ve, 


Concept of Usage 


SYSTEM CON-OP 


EXTERNAL 
OPERATIONS SEGMENT CON-OPS 


INTERFACES 
FUNCTIONAL IF’S SEGMENT 


MAJOR 
ACTIVITIES 


SEGMENT ROLE INSYSTEM 


OPERATIONAL FUNCTION OF 


OPERATIONAL 


FLOWS 
EACH SEGMENT SUBDIVISION 
INTERSEGMENT e HdW SUBSYSTEM 
OPERATIONAL s SIWCPCI 
STAFFING INTERFACES e POSITION 


CONTROL AND DATA INTERFACES 
TIMELINES 
WEEK 

MAINTAIN 

TRAIN INTEGRATE 


DAY 


PLAN 
—— 0.B. CHANGE 


- Secment i See 


| OPERATIONS 
Ll--——— 4 Key OPERATIONAL 
DESCRIPTIONS: 
a 4 e OPERATIONAL MODES 
e ACTIVITIES = - 
aN Cas DESCRIPTION 
| re FREQUENCY/DURATION 
); § HdW/SW/PE RSONNEL INVOLVED 
(ey no a SEQUENCE 


CRITICAL TIMING 
| OPERATIONAL CONSTRAINTS | 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


STATINTL 
HARDWARE REQUIREMENTS 


GENERAL REQUIREMENTS 


ENVIRONMENTAL COMPATIBILITY 
RELIABILITY/MAINTAINABILITY/AVAILABILITY 
OPERATIONAL SECURITY 

FAULT DETECTION & ISOLATION 
PRODUCIBILITY 

LOGISTICS SUPPORT 

TRANSPORTABILITY 

DESIGN STANDARDS 
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TATINTL 
HARDWARE REQUIREMENTS continued) 


ARCHITECTURE 


FUNCTIONALITY | 
FAILURE MODES & EFFECTS 
- TESTABILITY 


INTERFACES 


FUNCTIONAL RELATIONSHIPS 
ELECTRICAL CONNECTIONS 
SIGNAL CODING 
MECHANICAL CONNECTIONS 
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HARDWARE REQUIREMENTS (continued) 


FUNCTIONAL REQUIREMENTS 


LOADS 
SIGNALS 
ACCURACY 
TIMING 
CONTROL 
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TATINTL 


SOFTWARE REQUIREMENTS 


GENERAL REQUIREMENTS 


SECURITY/PRIVACY 
ARCHIVING/RETRIEVAL 
BACKUP/FALLBACK/RESTART 
DESIGN STANDARDS 


ARCHITECTURE 


COMPUTER ENVIRONMENT 
SCHEDULE/CONTRACTOR 
OPERATIONAL ENVIRONMENT 
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TATINTL 
SOFTWARE REQUIREMENTS (continued) 


INTERFACES 


DATA DESCRIPTION 
VOLUME 
MEDIA 


FUNCTIONAL REQUIREMENTS 


ACCURACY 
THROUGHPUT 
DETAILED TIMING 
CONTROL 
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OPERATOR REQUIREMENTS 


POSITION REQUIREMENTS 


DESCRIPTION OF ROLE 

TIMELINE 

WORKLOAD ANALYSIS 

SUPPORTING SUBSYSTEMS OR INFORMATION 


TRANSACTION REQUIREMENTS 


TASK DESCRIPTION 

TIMELINESS REQUIRED 

ACTIONS REQUIRED 

ERROR EFFECTS, CONTROLS & RECOVERY 
DECISION DATA AND BASIS 
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FACILITY REQUIREMENTS 


REQUIREMENTS 


ge POWER, 
COOLING 
SPACE 
STRUCTURAL 
SERVICES 
ACCESS 


~~ CONSTRAINTS 


SECURITY . 
LOGISTICS 
OPERATIONS 
SAFETY 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


TATINTL 
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TRADE-OFF OPPORTUNITIES 


TOTALLY DIFFERENT ALTERNATE APPROACHES 

MODIFIED ALLOCATION OF REQUIREMENTS BETWEEN SUBSYSTEMS 
VARYING LEVELS OF RELIABILITY, MEAN LIFETIME 

VARYING LEVELS OF PRESUMED THREAT LEVELS 

MANNED VS UNMANNED SYSTEMS 

FACTORY VS FIELD CALIBRATION 


USE OF DEMONSTRATION PROGRAM OR PROTOTYPE DEVELOPMENT 
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TATINTL 


DEVELOPMENT PLAN 


DEVELOPMENT TASK WBS 


PLAN 
SPECIFY 
DESIGN 
PRODUCE 
VERIFY 
DOCUMENT 


TASK SCHEDULES AND DEPENDENCIES 


MASTER TASK NETWORK 

MAJOR MILESTONE 

INTERNAL REVIEWS 

INTERNAL AND EXTERNAL DEPENDENCIES 
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TATINTL 


DEVELOPMENT PLAN (continued) 


STAFFING AND ORGANIZATION 


TASK RELATIONSHIP 
MANAGEMENT AND SUPPORT 
PROJECT/MATRIX ROLES 
TRAINING 


FACILITIES 


DOCUMENT PRODUCTION 
HARDWARE ASSEMBLY & TEST 
SOFTWARE DEVELOPMENT 
SECURE SPACE AND STORAGE 
TEST DATA GENERATORS 
ANALYSIS TOOLS 


MANAGEMENT & TECHNICAL CONTROLS 


COST REPORTING 

CRITICAL PERFORMANCE 
CONFIGURATION MANAGEMENT 
SECURITY 


a ee 
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QUALITY ASSURANCE j 


ESTABLISH DEVELOPMENT STANDARDS - 
MONITOR COMPLIANGE 

PROVIDE VENDOR PARTS INSPECTION 
AUDIT DELIVERED PRODUCTS 
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CONFIGURATION CONTROL 


IMPLEMENT MANAGEMENT CONTROL OF PRIORITIES 
PREVENTS UNNECESSARY CHANGES 
ASSURES INTERFACE COORDINATION 

"PROVIDES ORDERLY INTRODUCTION OF CHANGE 
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TATINTL 


REQUIREMENTS VERIFICATION 


REQUIREMENTS TRACEABILITY 
VERIFICATION PLAN & MATRIX 


INSPECTION 
ANALYSIS 
TEST/DEMONSTRATION 


TEST PLANS AND PROCEDURES 
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TATINTL 


TYPICAL DCR PROBLEMS 
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INCOMPLETE REQUIREMENTS ALLOCATION 


CAUSES 


INEXPERIENCED CONTRACTOR 
POOR ANALYSIS TOOLS 
INADEQUATE SCHEDULE OR BUDGET 


FAILURE TO CONSIDER DERIVED REQUIREMENTS 
CONSEQUENCES 


WASTED DESIGN EFFORTS 
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TAIN. [ 
INCOMPLETE OPERATIONS CONCEPT 


CAUSES 


INSUFFICIENT USER CONTACT 
INADEQUATE CONTRACTOR EXPERIENCE 
INSUFFICIENT RESOURCES 


CONSEQUENCES 


ERRONEOUS INTERFACES 
NON-TECHNICAL FACTORS IGNORED 
IMPRACTICAL TIMELINES 
UNREALISTIC TEST ENVIRONMENT 
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STAFFING DELAYS 


CAUSES 


CORPORATE PRIORITIES 

UNREALISTIC PLANNING ASSUMPTIONS 
CHANGING LABOR MARKET 
CLEARANCES 

PROJECT REPUTATION 


CONSEQUENCES 


IMPLEMENTATION DELAYS 
MORE MATURE PRECEDENT PRODUCTS 
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NON-OPTIMUM LABOR MIX 


CAUSES 


INEFFECTIVE MATRIX MANAGEMENT 
DYNAMIC BUSINESS BASE 


CONSEQUENCES 


UNABLE TO WORK AROUND SCHEDULE CHANGES 
AMPLIFIED COST IMPACT 
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TATINTL 


UNACHIEVABLE SCHEDULE 


CAUSES 


~POLITICAL PRESSURE 
REQUIREMENTS GROWTH 
POOR INTERFACE DEFINITION 
BUDGET REPROGRAMMING 
COMPETING PRQJECTS 


CONSEQUENCES 


POOR MORALE 

CONSTANT REPLANNING 
WASTED EFFORT 

COST & SCHEDULE GROWTH 
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DOCUMENT PRODUCTION DELAYS 


CAUSES 


UNDERESTIMATED WORKLOAD 
DELAYED INPUT 

LOW PROJECT PRIORITY 
INADEQUATE FACILITY 


CONSEQUENCES 


INADEQUATE REVIEWS 
MISSED MILESTONES 
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TATINTL 


REQUISITES FOR PROGRESS ASSESSMENT 


A PLAN 


TECHNICAL 
MANAGEMENT 


INFORMATION 


PROGRESS 
RESOURCES . 
CONDITIONS 


UNDERSTANDING OF INTERACTION 


BUDGET VS PROGRESS BY ELEMENT 
CRITICAL PATH ITEMS 
PRECEDENCE RELATIONSHIPS 
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TATINTL ——— - 
Quantified Measures | 


PRELIM, DESIGN CRITICAL DESIGN PRODUCTION 
REQ. ALLOC. & PROTOTYPE & DEVELOPMENT & TEST INTEGRATION 


DOCUMENTATION NO. OF RISK ITEMS 


NO. OF CRITICAL RACKS OF EQUIPMENT | NO. OF CIRCUITS UNITS COMPLETED 

REQUIREMENTS ‘ 
NO. OF STRUCTURES NO. OF INTERFACES TESTS COMPLETED 

NO. OF FUNCTIONS 
NO. OF DISPLAYS NO.OF INTERFACES UNITS ACCEPTED 

HAROWARE NO. OF SUBSYSTEMS 
7 NO. OF CABLES HOURS OF BURN-IN HOURS OF REASSEMBLY 

NO.OF OPERATING 

MODES NO.OF ASSEMBLY NO. OF ENCLOSURES NO. OF TESTS 
POSITIONS 

NO. OF TBD's/TBR’s NO. OF MECHANISMS TESTS EXECUTED 
NO. OF DRAWINGS 

PAGES OF ; TEST PASSED 

TESTS IN ANALYSIS 

NO. OF MENU's LINES OF CODE PATHS EXECUTED 

NO.OF CRITICAL REQ. NO. PEOPLE TRAINED 
DATA BASE SIZE NO. OF DATA ITEMS MODULES COMPLETED 

NO. OF FUNCTIONS 
NO.OF MODULES SPEED OF EXECUTION LINES OF CODE 

PRODUCED 
SOFTWARE NO. OF CPCI’s NO.OF CRITICAL WORDS OF MEMORY 

ALGORITHMS USED TEST DATA DEVELOPED 

NO. OF OPERATING 

MODES NO. OF SPEC. COMMENTS] NO. OF CHANGES TESTS EXECUTED 

NO. OF TBD’s/TBR’s NO. OF INTERACTIONS TESTS EXECUTED 

PAGES OF NO. OF DERIVED TESTS PASSED 

DOCUMENTATION REQUIREMENTS 


TESTS IN ANALYSIS 


NO. OF CRITICAL REQUNO. OF CRITICAL TASKS | NO. OF MANUALS NO. TRAINING SESSIONS] NO. PEOPLE TRAINED 
DESIGNED 
NO. OF POSITIONS NO. OF SLACK HOURS NO. OF CHAPTERS 
. NO. OF COPIES 

OPERATIONS] NO. OF OPS. NO. OF ACTIONS NO. OF PAGES PRODUCED 

SCENARIOS 

HOURS OF TRAINING NO. OF PEOPLE TO BE NO. OF PROCEOURES 
NO. OF FUNCTIONS TRAINED JEVELOPED 


ITEMS TO BE 
PAGES OF MAINTAINED 
DOCUMENTATION 


NO. OF INTERFACES : NO. OF ITERATIONS 
ON PLANS 


S INSTALLED NO. OF REQUIREMENTS 
NO. OF WALLS SATISFIED 
CONSTRUCTED 
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TATINTL 


Milestones 


SUBJECTS IDENTIFIED 
OBJECTIVES STATED 

APPROACH DEFINED | 

DATA IOENTIFIED 

DATA OBTAINED 

TOOLS IDENTIFIED 

TOOLS DEVELOPED 

OUTPUT RESULTS DEFINITIZED 
INTERMEDIATE STAGES DEFINED | 
STAGES ACCOMPLISHED 

RESULTS OBTAINED 


ANALYSIS CONCLUDED 


PURPOSE DEFINED 
AUDIENCE DESCRIBED 
OUTLINE COMPLETED 
OUTLINE APPROVED 

Is| DRAFT COMPLETED 
2nd ORAFT COMPLETED 
DRAFT DISTRIBUTED 
ORAFT COMMENTS RETURNED 
COMMENTS INCORPORATE 
FINAL DRAFT COMPLETED 
FINAL ORAFT APPROVED 
PRODUCTION COMPLETED 


DISTRIBUTION COMPLETED 


REQUIREMENTS DEFINED 


REQUIREMENTS APPROVED 


ARCHITECTURE DEFINED 


DEVELOPMENT APPROACHS 
PROPOSED 


DEVELOPMENT APPROACH 
SELECTED 


PRODUCTION TOOLS DEFINED 


FACILITY REQUIREMENTS 
DEFINED 


TEST REQUIREMENTS DEFINED 
DATA BASE DESIGNEO 
CONTROL LOGIC COMPLETED 


-DESIGN COMPLETEO 


PRODUCTION TOOLS AVAILABLE } 
INPUT/QUTPUT CODED 
PROTOTYPE COMPLETED 
PRODUCTION INITIATED 
PRODUCTION COMPLETED 

TEST INITIATED 

TEST COMPLETED 

PRODUCT SHIPPED 


PURPOSE DEFINED 


PARTICIPANTS IDENTIFIED 


OBJECTIVES DEVELOPED 


ENVIRONMENT OEFINED 


SEQUENCE DEVELOPED 


RESOURCES IDENTIFIED 


PARTICIPANTS NOTIFIED 


INSTRUCTIONS DEVELOPED 


RESOURCES PROVIDED 


ACTIVITY CONDUCTED 


ASSESSMENT COMPLETED 


LOGISTICS CONCLUDED 


eee eee (Se Se ee eT, 
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TATINTL 
| Earned Value Process (Cumulative) 
DCR PDR CDR Assembly rie Production lOc 


System Engineering 


Hardware Development 


Software Development 


System Test 
Firoduction 
Operations Support / 


Planned J 
ese i tr 240K [250K | -80K 410K | Staffing/Salaries | 
| 160K 80K 160K -80K }+80K | Clock Stability 
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TATINTL 


Balancing Progress 


100% 


EARNED 
VALUE 


MB hich risk 


TIME, $ EXPENDED HIGH RISK SELECTED . CONTINGENCY 
ACTIVITY INDICATORS PLAN 

MEASUREMENT PROTOTYPE ALTERNATE 

ACCURACY PERFORMANCE COMPONENT 
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DECISION 
BASIS 


SYSTEM 
PERF. vs$ 
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TATINTL 


Schedule Effects 


@ HIGH PROBABILITY OF EXTERNAL 


@ INDEPENDENT FUNCTIONS 
@ LARGE TEAM CHANGES 


@ MANY INTERFACES/ @ OPTIMUM SKILL MIX 


INTERACTIONS @ PERSONNEL TURNOVER 


@ FREQUENT SLACK @ SKILLS COMPROMISED 


= HIGH WASTE 


COMPRESSED BASELINE 
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TATINTL 


Development Interdependencies 


CONTRACTORA 


CODE 


DATA TRANSFER DESIGN 
NETWORK COMPLETED COMPLETED CODE TESTED 
PROTOCOLS \ 
KNOWN \ 
\ 
GRAPHICS . \ 
APHI 
DESIGN CODE 
CODE TESTED 
SENERAGOR COMPLETED COMPLETED 
CALLING 
\ SEQUENCES 
\ CODE NEEDED 
\ 


CONTRACTOR B 


APPLICATION REQMTS DESIGN CODE 
SOFTWARE COMPLETED COMPLETED COMPLETED SDE TESTED 


TIME —> 
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Bottlenecks & Interfaces 


FUNCTIONAL 
DESIGN 


INTERFACE 
COORDINATION 


DETAILED 
DESIGN 


PRODUCTION 


INTEGRATION | ir 


Limit Schedules 


DEVELOPMENT TIME, YRS 


PEAK MANPOWER 


: PERSON 10 PEOPLE 


MINIMUM POSSIBLE 
TIME 


~ 200 400 600 800 


LINES OF CODE (1000) 
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TATINTL 


PROGRESS REPORTS 


WORK REMAINING 


SUMMARY OF PLAN 
DETAILS OF DIVERGENCE 


RESOURCES AVAILABLE 


MONEY 

PEOPLE 

COMPUTER TIME 
HARDWARE AND FACILITIES 
TIME 


CORRECTIVE ACTIONS PROPOSED OR TAKEN 


MANAGEMENT 
TECHNICAL 
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~ | : 


How to Make Your Project Proceed . | e | 
Exactly According to Plan & 


ra 
* mee, =e 
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TATINTL | 
bor opsectives 


STABILIZE THE DESIGN ARCHITECTURE 

DESCRIBE TRADEOFFS AND SELECTION RATIONALE 
IDENTIFY KEY RISKS AND LONG-LEAD ITEMS 
ESTABLISH TOTAL PROJECT STATUS 

REFINE COST & SCHEDULE ESTIMATES 

CONFIRM OR REVISE DEVELOPMENT PLAN 
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TATINTL 


| pRELIMINARY DESIGN REVIEW AGENDA 
DCR PROBLEMS 


FUNCTIONAL FLOW 
ELECTRICAL SUBSYSTEMS 


_ CIRCUITS 
COMMUNICATIONS ENCODING 
POWER 


MECHANICAL SUBSYSTEMS 


MECHANISMS 
STRUCTURES 
EXPLOSIVE DEVICES 


DATA SUBSYSTEMS 


GENERAL 
COMPUTERS 

PERIPHERALS 

COMMUNICATIONS/DATA HANDLING EQUIPMENT 
OPERATOR STATIONS 
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TATINTL 


| PRELIMINARY DESIGN REVIEW AGENDA  ‘ntinued) 


SOFTWARE SUBSYSTEMS 


INPUT & OUTPUT 

CONTROL 

PROCESSING & KEY ALGORITHMS 
DATA BASE ARCHITECTURE 
TIMING AND STORAGE 


AVAILABILITY/RELIABILITY/MAINTAINABILITY 
INTERFACE REQUIREMENTS 

DEVELOPMENT PLAN REVISIONS 
MANUFACTURING 

PRELIMINARY TEST PLANS 


SUPPORT TOOLS 
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TATINTL . 7] 
| ERATIONAL REQUIREMENTS 


OPERATOR REQUIREMENTS 


FUNCTIONS 
RESPONSE TIME 
SKILL 

ERROR CONDITIONS 


OPERATIONAL CONSTRAINTS 


MUTUALLY EXCLUSIVE EVENTS 
MUTUALLY EXCLUSIVE OPERATIONAL REQUIREMENTS 
EXTERNAL INTERFACE DYNAMICS 
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STATINTL 


ELECTRICAL SUBSYSTEMS DESIGN cestinued) 


GENERAL 


SAFETY 
SECURITY 
CALIBRATION 
BUILT-IN-TEST 


POWER 


PRIMARY SOURCE 
STORAGE DECAY CHARACTERISTICS 
LOADING ANALYSIS 


CONTROLS 


RESPONSE CHARACTERISTICS 
STABILITY | 

ACCURACY 

FAILURE ANALYSIS 
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STATINTL 


ELECTRICAL SUBSYSTEMS DESIGN (continued) 
COMMUNICATIONS : | 


ERROR-DETECTION & CORRECTION TECHNIQUES 
AVERAGE & PEAK LOADING ° 
BUFFER REQUIREMENTS 
LINK MARGINS 


SPECIAL CIRCUITS 


SYSTEM RESPONSE PROPERTIES 
RELIABILITY ANALYSIS 
FAILURE MODES & EFFECTS 
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STATINTL ee | | % | 
RICAL SUBSYSTEMS DESIGN continued) 


SOLID STATE DEVICES 


SENSITIVITY 
DYNAMIC RANGE | 
SATURATION CHARACTERISTICS 


TRADE-OFF STUDIES 


WEIGHT vs LINK MARGIN 
SENSITIVITY vs SPATIAL COVERAGE 
POWER SHELF LIFE vs WEIGHT 
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STATINTL 
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MECHANICAL SUBSYSTEMS DESIGN 
GENERAL 
SAFETY MARGINS 


ENVIRONMENTAL PROTECTION 
OPERATING LIFE (CYCLES) 


STRUCTURES 


MATERIALS 

OADS ANALYSIS 
FAILURE ANALYSIS 
THERMAL DISTORTION 
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TATINTL 


MECHANICAL SUBSYSTEMS DESIGN _ tcontinued) 


MECHANISMS 


DESIGN APPROACHES 
CLEARANCES 

DYNAMIC ANALYSIS _ 
POWER REQUIREMENTS 


HYDRAULIC 


PRESSURE VESSEL DESIGN 
SEALS 

VALVES 

RELIABILITY 
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TATINTL 
MECHANICAL 
SUBSYSTEMS DESIGN 
THERMAL 


ENVIRONMENTAL ANALYSIS 
CONTROLS 7 


PYROTECHNIC 
SAFETY 


STABILITY 
RELIABILITY 
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STATINTL 


DATA SUBSYSTEMS DESIGN 


GENERAL 


MTBF/MTTR 
NOISE/HEAT/AIR VELOCITY 
EMI 

FAULT DETECTION 
BUILT-IN-TEST 


COMPUTERS 


INSTRUCTION SET 
EXECUTION RATE 
I/O CHANNELS 
WORD LENGTH 
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STATINTL | | 
ATA SUBSYSTEMS DESIGN continued) 


PERIPHERALS | 


I/O RATE 
VOLUME 
SEEK TIME 
PRINT RATE 
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TATINTL 


DATA SUBSYSTEMS DESIGN  Ccontinueg) 


-COMMUNICATIONS/DATA HANDLING EQUIPMENT 


BANDWIDTH/DATA RATE 

WORD LENGTH @ 

ENCODING METHOD 

NUMBER OF CHANNELS 

STATUS MONITORING 

ERROR DETECTION AND CORRECTION 
BIT ERROR RATE 

FAILURE RECOVERY MODES 
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DATA SUBSYSTEMS DESIGN (continued) 


~ OPERATOR STATIONS 


FUNCTIONS 

INPUT DEVICES 

DISPLAY CHARACTERISTICS » 
RESPONSIVENESS 


TRADE-OFF STUDIES 


BUILT-IN-TEST vs DEPOT MAINTENANCE 
NET ERROR RATE vs NET DATA RATE 
TRAINING TIME vs WORK LEVEL THROUGHPUT 
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TATINTL 
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[ 


SOFTWARE SUBSYSTEMS DESIGN 


GENERAL 


LANGUAGE 

PROGRAMMING STANDARDS 
COMMERCIAL HARDWARE 
COMMERCIAL SOFTWARE 
CONCURRENT REQUIREMENTS 
SECURITY/PRIVACY 


INPUT 


DESCRIPTION 
UNITS 


‘SOURCE 


RANGES/LIMITS 
ACCURACY/PRECISION 
FREQUENCY/VOLUME 
MESSAGE FORMATS 
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TATINTLE teh Wage Se a8 Sac re ee manent 


SOFTWARE SUBSYSTEMS DESIGN _ Continuea) 


PROCESSING 


FUNCTIONAL DESCRIPTION 
CONTROL OPTIONS 

, COORDINATE SYSTEMS 
ACCURACY 
SEQUENCE OR TIMING ~ 
KEY ALGORITHMS 

_ DATA BASE DESIGN STRUCTURE 
STIMULI AND RESPONSE 
THROUGHPUT 
RESPONSE TIME 


OUTPUT 


DESCRIPTION | 
DESTINATION 
UNITS 
ACCURACY/PRECISION 
FREQUENCY/VOLUME 
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SOFTWARE SUBSYSTEMS DESIGN Continued) 


CONTINGENCY RESPONSES 


ERROR DETECTION/CORRECTION 
BACKUP 

FALLBACK 

RESTART 


MAN/MACHINE INTERFACE 


CAPABILITY 
RESPONSE TIME 
DISPLAY DENSITIES/COLORS 


TRADE-OFF STUDIES 


RE-COMPUTATION TIME vs STORAGE 
ACCURACY vs TIMING 

MENU DRIVEN vs COMMAND DRIVEN 
ON-LINE vs OFF-LINE ARCHIVING 
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DESIGN METHOD DIFFERENCES 


HARDWARE 


PIECE PARTS AS BUILDING BLOCKS 
STANDARD PARTS AVAILABLE 


COMPONENTS 
CONNECTORS 


TECHNOLOGY DIFFERENCES WELL QUANTIFIED 


MOS/BIPOLAR/CCD 
HONEYCOMB/COMPOSITES 
_ PCM/FDM/AM 


NEW PARTS ARE DESIGNED WITH MULTIPLE APPLICATIONS IN MIND 


FEW DECISIONS MADE ‘“‘ON THE ASSEMBLY LINE” CAN EFFECT 
PERFORMANCE 
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STATINTL 


DESIGN METHOD DIFFERENCES continued) 


SOFTWARE 


ROUTINE IS THE BUILDING BLOCK 
ONLY BASIC MATH BUILDING BLOCKS 


TRIG FUNCTIONS 

SOME STATISTICAL AND SIGNAL ANALYSIS PACKAGES 
LIMITED DATA BASE OPTIONS 

AVAILABILITY VARIES WITH HOST MACHINE MARKET SIZE 


TECHNOLOGY DIFFERENCES HOTLY DEBATED 


ADA vs JOVIAL vs FORTRAN 
RELATIONAL vs HIERARCHICAL DATA STRUCTURES 


NEW ROUTINE ARE DESIGNED ONLY FOR THIS APPLICATION 
MANY DECISIONS MADE “ON THE ASSEMBLE LINE” EFFECT 
PERFORMANCE 
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TATINTL 


PRELIMINARY TEST PLANS 


TEST PROGRAM PHILOSOPHY & STRUCTURE 
TEST DESCRIPTIONS 
SUCCESS CRITERIA 
TEST SEQUENCE 

_INPUT AND ANALYSIS PREPARATION 
RESOURCES 
CONTROLS AND REPORTING 
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STATINTL 


TYPICAL PDR PROBLEMS 
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STATINTL [ay 
| OR INTERFACE DEFINITION 


CAUSES 


WEAK INTEGRATOR 
ASYNCHRONOUS SCHEDULES 
PROFIT MOTIVE 


CONSEQUENCES 


DESIGN GAPS 
SCHEDULE SLIP 
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INCOMPLETE SYSTEMS ENGINEERING 


CAUSES | 
BUDGET SQUEEZE 
INADEQUATE SOW 
UNAGGRESSIVE LEADERSHIP 
CONTRACTOR INEXPERIENCE 
ERROR SOURCES NOT RECOGNIZED 


CONSEQUENCES 


UNATTAINABLE ACCURACY SPECIFICATIONS 
DEGRADED AVAILABILITY 
INCREASED FAILURE IMPACT 
MISSED TIMELINES 
DIMINISHED SENSITIVITY 
LOST DATA 
INVALID ARCHITECTURE 
LATE REQUIREMENTS ADDED 
COST AND SCHEDULE GROWTH 
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STATINTL? 


INCOMPLETE OPERATOR POSITION DEFINITION 


CAUSES 


INSUFFICIENT USER CONTACT 
UNSKILLED CONTRACTOR 


CONSEQUENCES 


REDUCED SYSTEM THROUGHPUT 
LATE DEFINITION OF OUTPUT REQUIREMENTS 
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STATINTL 
UNIDENTIFIED LONG-LEAD ITEMS 


CAUSES 
INCOMPLETE TEST PLANNING 
INCOMPLETE PRODUCTION PLANNING 
GOVERNMENT FURNISHED DATA 
CONSEQUENCES 


COST & SCHEDULE GROWTH 
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STATINTL 


POOR QUALITY DOCUMENTATION 


CAUSES 
LACK OF EXPLICIT GUIDANCE 
UNREALISTIC SCHEDULE 
INEFFECTIVE INTERNAL REVIEWS 
BAD WRITERS 

CONSEQUENCES 


DIFFICULT TRANSITION BETWEEN PHASES 
COST & SCHEDULE GROWTH 
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STATINTL [| 
INADEQUATE SUBCONTRACT VISIBILITY 


CAUSES 


CONTRACT FLAWS 
PASSIVE GOVERNMENT PROGRAM OFFICE 
CONTRACTOR STRATEGY 


CONSEQUENCES 


MISSED OPPORTUNITIES 
REACTIVE GOVERNMENT ROLE 
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TATINTL 


UNINFORMATIVE PROGRESS REPORTS 


CAUSES 


QUALITATIVE REPORTING 

NO COMPARISON TO A PLAN 
TOO HIGH A LEVEL OF SUMMARY 
TOO MUCH DETAIL 

OUTDATED INFORMATION 


CONSEQUENCES 


HIDDEN PROBLEMS 
COST AND SCHEDULE GROWTH 
COUNTERPRODUCTIVE ACTIONS 
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TATINTL 


REQUIREMENTS RISK 


SOURCES 


ILL-DEFINED MISSION 
LACK OF USER CONSENSUS 
POORLY UNDERSTOOD ENVIRONMENT 


CLUES 


CONFLICTING SUB-MISSION PRIORITIES 


DIVERSE OPINIONS REGARDING SOLUTION CHARACTERISTICS 
LACK OF DEFINITION 


REDUCTION 


USER COORDINATION 
ENVIRONMENTAL SAMPLING 
CONCEPT DEFINITION PHASE 
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STATINTL 


TECHNICAL RISK 


SOURCES 


PHYSICAL PROPERTIES 

DESIGN INSTABILITY 
UNREALISTIC PERFORMANCE 
UNPROVEN LONG TERM EFFECTS 
SHORTSIGHTEDNESS 


CLUES 


LAGGING DEFINITION OF PROJECT ELEMENTS 
UNCALIBRATED ASSUMPTIONS 
CONTRARY OPINIONS 


REDUCTION 


DESIGN REVIEWS 
PROTOTYPES 
MULTIPLE PROCUREMENTS 
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TATINTL 


MECHANICAL SYSTEMS 


SOURCES 


MATERIAL PROPERTIES 
ENVIRONMENTAL RESPONSE 
SAFETY MARGIN - 

DYNAMIC DISTORTION 


ESTABLISHING EXTENT 


LABORATORY TESTS 
ENVIRONMENTAL SAMPLING 
ENGINEERING ANALYSIS 
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TATINTL 


ELECTRONIC SYSTEMS 


SOURCES 


COMPONENT STABILITY 
PACKAGING DENSITY 
PRODUCTION YIELDS 
ENVIRONMENTAL AGING 


ESTABLISHING EXTENT 


LABORATORY TESTS 
PROTOTYPE DEVELOPMENT 
ENVIRONMENTAL TESTS 
QUALITY CONTROL SYSTEMS 
COMPUTER AUTOMATED DESIGN _ 
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aia | 
MPUTERS 


SOURCES 
RESOURCE CONTENTION 
RESPONSE UNDER LOAD 
SOFTWARE RESOURCE DEMANDS 
ESTABLISHING EXTENT 
BENCHMARK TESTING 


DETAILED SIMULATION 
USER SURVEYS 
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SOFTWARE 


SOURCES 


COMPILER/OPERATING SYSTEM MATURITY 
COMPUTER ARCHITECTURE COMPATIBILITY 
PROBLEM DOMAIN KNOWLEDGE 
UNREALISTIC PERFORMANCE LEVELS 


ESTABLISHING EXTENT 


~ SOFTWARE USER SURVEYS 
CONTRACTOR SITE OBSERVATIONS 
DETAILED SIMULATION 
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SOFTWARE vs HARDWARE 


CATCH-ALL FOR PROBLEMS 


CANNOT BE SOLVED IN HARDWARE 

DON’T KNOW HOW TO IMPLEMENT IN HARDWARE 
TOO ILL-DEFINED FOR HARDWARE 

CATCH-UP FOR EARLY DESIGN OVERSIGHTS 


REQUIRE KNOWLEDGE OF OTHER “‘DESIGNS’”’ 


DATA COLLECTOR OPERATIONAL PARAMETERS 
REAL-TIME DATA CORRUPTION 

OPERATOR ‘‘PREFERENCES” 

EXISTING ACCOUNTING PROCEDURES 
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SOFTWARE vs HARDWARE (continued) 


MORE INTERFACES PER MODULE 


HARDWARE 
CONNECTORS 
SWITCHES | “ 
(USUALLY) WELL DEFINED INPUT/OUTPUT 
SOFTWARE 
OPERATOR CONTROLS 
OPERATING MODES 
DISPLAY OPTIONS 
MYRIAD OF DATA INTERFACES 
EMBEDDED INTERFACES (DATA-DRIVEN BRANCHES) 
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SOFTWARE vs HARDWARE ss ‘ontinued) 


INTANGIBLE DESIGN 


CANNOT BE TOUCHED OR SEEN 
CONCEPTUAL IN NATURE 
“EASY TO CHANGE” 


YOUNGER TECHNOLOGY 


CHANGING STANDARDS & CONVENTIONS 
CHANGING DESIGN METHODS 
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COST & SCHEDULE RISK 


SOURCES 


REQUIREMENTS & TECHNICAL 
CONTRACT TERMS 
CONTRACT DELAYS 
MANAGEMENT 
UNCONTROLLABLE 


CLUES 


FREQUENT CHANGES TO PLANS 
NO CHANGES TO PLANS 

UNDER OR OVER STAFFING 
MISSED MILESTONES 


REDUCTION 


SPECIAL AUDITS 
SUBCONTRACTS 


PHASFN DELIVERIES 
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[ 
CDR OBJECTIVES 


VALIDATE DESIGN APPROACH AND STANDARDS 

AFFIRM TEST, INTEGRATION AND TRANSITION RESOURCES 
ASSURE ADEQUATE TEST PROGRAM 

CALIBRATE DEVELOPMENT PROGRESS 
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STATINTL 


CRITICAL DESIGN REVIEW AGENDA 


PDR PROBLEMS 
PRODUCT SPECIFICATIONS 


DETAILED FLOW DEFINITION 
PROCESS DESCRIPTIONS 
SCHEMATIC DIAGRAMS 
PROTOTYPE RESULTS 


DATA BASE SPECIFICATION 


FILE STRUCTURE 
ITEM DESCRIPTIONS 
INDEXING/LINKAGE 


INTERFACE SPECIFICATIONS 


ENCODING 
PROTOCOLS 
_ CONNECTOR DESIGNS 
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ASSEMBLY TOOLING 
PARTS INSPECTION RESULTS 
TEST PLANS 


TEST RELATIONSHIPS | 
DATA SOURCES 

TEST EQUIPMENT 
PRELIMINARY PROCEDURES 
ACCEPTANCE CRITERIA 


INSTALLATION DRAWINGS 
RELIABILITY 

SAFETY 

SUPPORT DOCUMENTATION 


EQUIPMENT OPERATION MANUALS 
LOGISTIC SUPPORT PLANS 
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aise ——=$—-- 1 
HARDWARE SPECIFICATIONS 


ELECTRICAL 


CIRCUIT LAYOUTS 
POWER BUSSES & GROUNDING 
SHIELDING DESIGN 


MECHANICAL 


FINAL DRAWINGS 
FINAL PACKAGING 
FINISH INSTRUCTIONS 


THERMAL CONTROL 
COATING SPECIFICATIONS 
ELECTRO-MECHANICAL INTERFACES 


CONNECTORS 
LINKAGES 
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HARDWARE SPECIFICATIONS (continued) 


MASS PROPERTIES 


SIZE 
WEIGHT 
INERTIAL PROPERTIES 


PACKAGING & TRANSPORTATION 


SHOCK & VIBRATION MOUNTINGS 
THERMAL WRAPS 
STATIC DISCHARGE WRAPS 


RELIABILITY/MAINTAINABILITY/AVAILABILITY 


PARTS INSPECTION PROGRAM 
ENVIRONMENTAL TESTING 


SAFETY/SECURITY 


SWITCH COVERS 
DESTRUCTIVE DEVICES 
SHIELDING 
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DESIGN ANALYSIS & TEST DATA 


DETAILED LOGIC DIAGRAMS 
CIRCUIT DIAGRAMS 
PROCESSING ALGORITHMS 
TIMING ANALYSIS 
INPUT/OUTPUT DATA LINES 
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DESIGN RELIABILITY 


LOADS ANALYSIS 
FAILURE MODES & EFFECTS ANALYSIS 
THERMAL ANALYSIS ~ 
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PRODUCIBILITY 


MANUFACTURING TECHNOLOGY 
ASSEMBLY TOOLING 

ASSEMBLY PLAN 

PARTS AVAILABILITY 

QUALITY CONTROL 
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SOFTWARE SPECIFICATIONS 


INPUT 


NAME 
FORMAT 
DEVICE 
PRIORITY 


STATIC DATA 


NAME 

DEVICE 

NUMBER OF RECORDS OR ENTRIES 
DEFINITIONS 
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SOFTWARE SPECIFICATIONS (continuea) 


CPC DESIGN 


LOGIC 

DATA FLOW 

ERROR CHECKS 

ERROR MESSAGES/RESPONSES 
EQUATIONS 

ALGORITHMS 

INTERNAL VARIABLES 

CALLING RELATIONSHIPS 
BUFFERS/TEMPORARY STORAGE 
INTERRUPTS 
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STATINTL 


SOFTWARE SPECIFICATIONS (continued) 


GRAPHIC REPRESENTATION 


SEQUENCE 
ITERATION 
BRANCHES 


OUTPUT 


DISPLAY FORMATS/MEANS 
RECIPIENTS/NOTIFICATION 
FILE FORMAT 

STORAGE DEVICE 


LIMITATIONS OR CONSTRAINTS 


MEMORY AND STORAGE ALLOCATIONS 
CRITICAL TIMING 
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STATINTL 


DATA BASE SPECIFICATION 
LOGICAL DESCRIPTION | 


FILE-RECORD-FIELD DESCRIPTIONS 
USE 

FORMAT DEFINITIONS 

POINTERS AND CHAINS 

MARKERS 


FUNCTIONAL DESCRIPTION 


SET/USE 
ARCHIVING 
PURGING 


PHYSICAL DESCRIPTION 


FILE SIZE 

DEVICE ALLOCATIONS 

RECORDING FORMATS, DENSITY, PARITY 
PASSWORD ACCESS 
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SEGMENT DESIGN INTERFACES 


SOFTWARE/DATA BASE 


DATA TRANSFERS 
" REDUNDANCY/ARCHIVING 
PRIVACY 


SOFTWARE/HARDWARE 


CONTROL 
COMPUTATIONAL RESOURCES 


OPERATIONS/SOFTWARE 


DATA ENTRY AND RETRIEVAL 
SYSTEM OPERATIONS CONTROL 
SOFTWARE EXECUTION LEVELS 


OPERATIONS/HARDWARE 


DISPLAY AND CONTROL DEVICES CHARACTERISTICS 


MESSAGE AND ALARM CHARACTERISTICS. 
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REQUIREMENTS VERIFICATION 


_ SPECTRUM OF METHODS AND ACTIVITIES 


TESTING 


PITFALLS 
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The Verification Spectrum 


USERS 
PROCEDURES 
SYSTEM TRAINING 
REQUIREMENTS 
CI/CPCI & EXTERNAL 
TEST & OEBUG SEGMENT INTERFACES 
REQUIREMENTS 
COR SHIP INTEGRATION TRANSITION 10C 
TIME > 
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SCOPE -> 
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REQUIREMENTS VERIFICATION METHODS 


INSPECTION 


TRAINING PROGRAM 
EQUIPMENT SURFACE FINISH 
MODULARITY 


ANALYSIS 


AVAILABILITY 
ACCURACY 


TEST 


RESPONSE TIME UNDER LOAD 
OPERATING MODES 
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| The Three Faces of Testing 


QUALITY CONTROL REQUIREMENT VERIFICATION 


INSPECTION 
AUDITS 


PROGRAMMING 
STANDARDS 


STRUCTURED 
LANGUAGES 


ENGINEERING ANALYSIS 


DESIGN STANDARDS 


MATERIAL 
INSPECTION 


CLASSROOM 
TRAINING 


DEMO'S 
REHERSALS 


OJT & CERTIFICATION 


TRAINING 
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EVOLVING TEST ACTIVITIES 


CODE AND DOCUMENTATION REVIEW 
BENCH TEST RESULTS 
ANALYSIS RESULTS 
CONTROLLED TESTS 
KNOWN INPUT 


SIGNAL PROFILE 
TORQUE 
CHECK NUMBER 


DETERMINISTIC OUTPUT 


ENERGY MEASURE 
YIELD STRAIN 
AMOUNT PAYABLE 
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EVOLVING TEST ACTIVITIES 


(Continued) 
DEMONSTRATION TESTS 
CALIBRATED INPUT 
DYNAMIC LOADS PROFILE 
SIGNAL ENVIRONMENT TAPE 
DAILY ACCOUNT ACTIVITY 
BOUNDED RESULTS 
DEFLECTION HISTORY 


REPORTS TIMELINE 
SYSTEM RESPONSE TIME 
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TEST ACTIVITIES AND PROCEDURES 


REQUIREMENTS SATISFACTION 


DATA BASE ADEQUACY 
PRIOR CALCULATIONS AND BASELINES 
PARTIAL SATISFACTION 


TEST PROGRAM MANAGEMENT 


TEST TEAM STAFFING 

SCHEDULING FIXED RESOURCES 

TEST SEQUENCE PRECEDENCE RELATIONSHIPS 
TEST REQUIREMENTS SATISFACTION MATRIX 


FINAL AUDITS 


PRIORITIZE PRODUCTS 
PROPORTIONAL TO TEST INDUCED CHANGES 


LIENS AND WAIVERS 
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TEST SCHEDULING PITFALLS 


OPTIMISM 


FAILURE RATE 
PROBLEM ISOLATION 
TIME TO REPAIR 
ANALYSIS TIME 


OVERSIGHTS 


RECONFIGURATION TIME 
COMPUTER AVAILABILITY 
FACILITY OPERATIONAL REQUIREMENTS 


EFFICIENCY 


RESCHEDULING RESPONSE TIME 
PERSONNEL AVAILABILITY 
RECOMPILE/RECONNECT TIME 
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SEGMENT TEST PLAN 


TEST PROGRAM PHILOSOPHY AND STRUCTURE 


FUNCTIONAL/INTERFACES 
RE-TESTING e 
DRIVERS/STUBS . 
ON-SITE TESTING 

BUILD SEQUENCE 


TEST DESCRIPTIONS 


RANGE OF CONDITIONS 
TEST DATA OR ENVIRONMENT 
~ ACCEPTANCE CRITERIA 
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SEGMENT TEST PLAN (continued) 


TEST FLOW 


INTERRELATIONSHIPS 
DEPENDENCIES 


IMPLEMENTATION PLAN 


EXTERNAL DATA & CONTINGENCY PLANS 
SCHEDULE 
EQUIPMENT/SOFTWARE/PERSONNEL NEEDED 


MANAGEMENT CONTROLS 
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SEGMENT TEST PROCEDURE 


TEST OBJECTIVES 


GENERAL 
MEASURABLE 


~ OPERATING PROCEDURES 


SWITCH SETTINGS 

TAPE MOUNTING 

POSITION MANNING AND FUNCTION 
OPERATING SEQUENCE 

OPERATOR INTERVENTION INSTRUCTIONS 
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SEGMENT TEST PROCEDURE 


DETAILED DESCRIPTION 


INTERFACE REQUIREMENTS 
TEST SCENARIO 
OPERATOR DATA REQUIREMENTS 
TEST DATA BASE DEFINITION 
- SIMULATED INPUT DATA 
OPERATING MODE DEFINITION 
OBSERVABLE EVENTS - SEQUENCE AND DEPENDENCY 
INSTRUMENTATION OR RECORDING REQUIREMENTS 


ANALYSIS PROCEDURES 
DATA REDUCTION OR SUMMARY FORMATS 
STATISTICAL METHODS AND EQUATIONS 
REQUIRED DATA CONVERSIONS OR HAND CALCULATIONS 


ACCEPTANCE CRITERIA 
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CCEPTANCE CRITERIA 


WHAT SPECIFIC TANGIBLE EVIDENCE CAN BE PRODUCED 
CAN THE INTERPRETATION BE AMBIGUOUS? | 

IS SATISFACTION DIRECTLY OBSERVED OR IS IT IMPLIED 
ARE GUIDELINES FOR INPUT CONDITIONS COMPLETE? 


CAN UNSATISFACTORY PERFORMANCE PASS 
- ACCEPTANCE? | 


WHAT MINIMUM PERFORMANCE WOULD BE ACCEPTED BY 
A COURT? 
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INTERFACE DEFINITION PROBLEMS 


STATEMENT OF WORK CONFLICTS 
CONTRACT TYPE DIFFERENCES 
TECHNICAL AMBIGUITIES 
SCHEDULE SYNCHRONIZATION 
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CHANGE CONTROL 


CONTROL RESOURCE PRIORITIES 

PREVENT UNNECESSARY CHANGES 

ASSURE INTERFACE COORDINATION 

PROVIDE ORDERLY INTEGRATION OF CHANGE 
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_ Configuration Control Authority 


ARCH. 
AOC REVIEW POR COR SHIP 1QC 


SYSTEM/SEGMENT SPEC Re 


INTERFACE CTRL DOC 
DEVELOPMENT OPS CONCEPT 


Vi bdhvvvlreeess 


SEGMENT DESIGN SPEC 
SEGMENT OPS CONCEPT 
DEVELOPMENT SPECS 
PRODUCT SPECS 

SEGMENT INTRAFACE SPECS 


OO) 
“Be By ey) eeeeees 


STILL 
OLDE 


SEGMENT TEST PLAN 

SYS. INTERGR. TEST PLAN 
TRANSIT. INTERGR. PLAN 
SEGMENT TIP ~ 
SEGMENT LOGIS. PLANS 


MA MOO 
Wii 


USER MANUALS Wiilsshé“«ssh 


OPERATIONS/MAINT. MAN. a 
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FACILITY REQUIREMENTS 


AREA 
POWER 
COOLING & CIRCULATION 


AIR 
WATER 


COMMUNICATIONS 
ELECTROMAGNETIC RADIATION 
SOUND 

INTERCONNECTIONS 
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STATINTL 
INSTALLATION ISSUES 


MECHANICAL 


STRUCTURAL SUPPORTS 
ACCESS CLEARANCE 
MANUFACTURING TOLERANCES 


ELECTRICAL 


CABLE LENGTHS AND ATTENUATION 
ISOLATION/SHIELDING 

SWITCHES 

MODULATION, ENCODING, POLARITY 
GROUNDING | 


ENVIRONMENTAL 


CONTROL OPTIONS 
RANGES 
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FACILITY MODIFICATION CONCERNS 


IMPACT DURING MODIFICATION 
FUNCTIONALITY IN OPERATION 
SYSTEM DESIGN IMPACT 

LEAD TIME & INTERDEPENDENCES 
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SITE TESTING 


COMPLETE SEGMENT TESTS AS REQUIRED 


VALIDATE SYSTEM INTERFACES 
LAGGING PROBLEM FIXES 


DEMONSTRATE KEY REQUIREMENTS SATISFACTION 


UNIFY HARDWARE, SOFTWARE, PERSONNEL, PROCEDURES 
AND LOGISTIC SUPPORT 
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STATINTL 


SITE TESTING PITFALLS 


STRUCTURAL DEPENDENCE ON DELIVERY 
SCHEDULES 

DATA SOURCES 

CONTRACTOR MAINTENANCE 

OPERATIONAL IMPACT 
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STATINTL 
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[| 
TRANSITION TEST OBJECTIVES 


ESTABLISH SYSTEM PERFORMANCE WITHIN OPERATIONAL 
FRAMEWORK 


PERSONNEL 
PROCEDURES 
FACILITIES 
LOGISTICS 


CONFIRM TOTAL SYSTEM CAPABILITY 


OPERATING MODE CONTINUITY 
THROUGHPUT 

TIMELINESS UNDER LOAD 

ERROR DETECTION AND RECOVERY 


RECONFIRM CAPABILITIES NOT INTENDED TO CHANGE 
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STATINTL 


TRANSITION TEST PLANNING PROBLEMS 


LIEN IMPACTS ON TEST OBJECTIVES 
OPERATIONAL IMPACTS 


PERSONNEL 
EQUIPMENT UTILIZATION 
DATA BASE CONTROL 


OPERATIONAL RISKS 


FAILURE ISOLATION 
RECOVERY CAPABILITY 
DATA INTEGRITY 
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TATINTL 


TRANSITION TEST PLANNING PROBLEMS 


PROBLEM ISOLATION ANALYSIS 


OPERATIONAL ROLE 
DEVELOPMENT CONTRACTOR ROLE 


SIMULATED INPUTS AT EXTERNAL INTERFACES 
UNCONTROLLED LIVE SYSTEMS AND DATA 


POSTMORTEM 
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(continued) 
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TATINTL a | —d 


USER 


TRAINING 


CAPABILITIES 
ARCHITECTURE 
PRODUCTS 
APPLICATIONS 


OPERATOR 
OPERATIONAL POSITIONS 


FUNCTIONS 
INPUTS 
OUTPUTS 
CONTROLS 
DISPLAYS 


COMPUTER SYSTEMS 

DATA MANAGEMENT 
SPECIALIZED HARDWARE 
CONSTRAINTS OR LIMITATIONS 
USE OF DOCUMENTATION 
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TATINTL 


TRAINING (continued) 
MAINTENANCE 


* DIAGNOSTIC TOOLS/SELF TEST 
FAILURE MODE INTERACTIONS & SYMPTOMS 
USE OF DOCUMENTATION 


PLANNING FACTORS 


NUMBER AND AVAILABILITY OF TRAINEES 
RECERTIFICATION REQUIREMENTS 
EQUIPMENT AVAILABILITY 
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TATINTL 


LOGISTIC SUPPORT ELEMENTS 


CONTINUING TRAINING AND CERTIFICATION 


SYSTEM TRAINING 
POSITION TRAINING 
RECERTIFICATION 


MAINTENANCE 


HARDWARE PREVENTATIVE MAINTENANCE 
HARDWARE REPAIR 

SOFTWARE MODIFICATION 

SOFTWARE REPAIR 


SPARES & SUPPLIES 
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MEMORIZE YOUR CONTRACT 


REQUIREMENTS 
STATEMENT OF WORK 
TERMS AND SCHEDULES 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


MAINTAIN USER CONTACT 


KNOW HIS NEEDS 

KNOW YOUR CONTRACTOR'S CAPABILITIES 
UNDERSTAND COST/SCHEDULE/PERFORMANCE TRADES 
SELL YOUR PRODUCT 
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NURTURE PEOPLE 


VALUABLE INFORMATION SOURCES 
LONG-TERM GOVERNMENT BENEFITS 
PERSONAL SATISFACTION 
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LOOK FOR ASSUMPTIONS 


MEANING OF WORDS 
GROUND-RULES 
SIDE EFFECTS 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


Approved For Release 2002/08/19 : CIA-RDP88R00729R000200160001-9 


BE FORWARD LOOKING 


TOTAL PROJECT IMPACT 


DESIGN 
PRODUCTION 
TEST 
LOGISTICS 


RIPPLE EFFECTS 


DESIGN CHANGES 
STAFFING 
MILESTONE CHANGES 
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ANTICIPATE 


CONSIDERING: 


YOUR MISSION REQUIREMENTS 

STATE OF THE ART OF TECHNOLOGY BEING USED | 
YOUR CONTRACTOR’S CAPABILITIES 

HIS MANAGEMENT SKILLS 


ASK YOURSELF: 


WHAT NEEDS TO BE DONE TO MEET THE NEXT 
MILESTONE? 


WHO'S DOING IT? 

DO YOU HAVE REASONS TO THINK THEY’LL SUCCEED? 
HOW WILL YOU CALIBRATE THEIR PROGRESS? 

DO YOU HAVE ANY ALTERNATIVE OPTIONS? 
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PRELIMINARY TEST PROCEDURES 


TEST OBJECTIVES 

EQUIPMENT AND STAFFING 
OPERATING PROCEDURES 
DETAILED DESCRIPTION 
ANALYSIS PROCEDURES 
ACCEPTANCE CRITERIA 
RELATIONSHIP TO OTHER TESTS 
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Se a ne 


TATINTL 


TRANSITION INTEGRATION PLAN 


ORGANIZATIONAL ROLES AND MANAGEMENT SYSTEMS 
SCHEDULES 


IMPACTS 
DEPENDENCIES 


BASELINE EVOLUTION 


BASELINE CHANGE SEQUENCE 
INCREMENTAL FUNCTIONAL FLOWS 


FACILITY OVERVIEW 
OPERATIONAL RESOURCE DEMANDS 


PERSONNEL 
INTERFACES 
SYSTEMS 
MISSION IMPACT 
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TYPICAL CDR PROBLEMS 
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DESIGN INCONSISTENCIES 


CAUSES 


POOR INTERNAL COMMUNICATIONS 
AMBIGUOUS INTERFACE SPECIFICATIONS 
INEFFECTIVE MANAGEMENT 


CONSEQUENCES 


WASTED EFFORT 
SEGMENT TEST DELAYS 
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STATINTL 


INSUFFICIENT COMPUTER TIME 


CAUSES 


UNDERSIZED SYSTEM 
MISMANAGED RESOURCE 

LOW PRIORITY 

DEVELOPMENT PLANNING ERRORS 


CONSEQUENCES 


IMPLEMENTATION AND TEST DELAYS 
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STATINTL [ , 
INADEQUATE TEST FACILITIES 


CAUSES 


UNRECOGNIZED COMPLEXITY 
INEXPERIENCED TEST MANAGER 
LACK OF CORPORATE SUPPORT 


CONSEQUENCES - 


UNUSABLE SYSTEM 
INTEGRATION PROBLEMS 
PROJECT DELAYS 
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UNSPECIFIED TEST DATA SOURCES 


“ 


CAUSES 


LATE STAFFING 
UNRESOLVED TEST TOOLS 


CONSEQUENCES 


LONG LEAD TIME SCHEDULE IMPACT 
GOVERNMENT INHERITED PROBLEM 
LESS RIGOROUS TEST PROGRAM 
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TATINTL | 
SCHEDULE AND DESIGN BREAKAGE OF TEST PLANS 


CAUSES 


DEVELOPMENT MILESTONE SLIPS 
ENGINEERING OVERSIGHTS 
REQUIREMENTS CHANGES 


CONSEQUENCES 


WASTED EFFORT 
COMPRESSED TEST SCHEDULE 
INCREASED INTEGRATION TESTING 
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TATINTL 


UNTESTABLE DESIGN 


CAUSES 
~ TEST APPROACH UNSPECIFIED AT PDR 
CONSEQUENCES 


INTEGRATION RISK 
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IGNORING EVIDENCE OF A SLIP 


CAUSES 


CUSTOMER WON’T ACCEPT TRUTH 
MANAGEMENT PRESSURE 
WAITING FOR ‘“‘GET-WELL” 


CONSEQUENCES 


CONTINUED SLIPS 
POOR MORALE 
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ene ree a 


CONFIGURATION CONTROL 


IMPLEMENT MANAGEMENT CONTROL OF PRIORITIES 
PREVENTS UNNECESSARY CHANGES 

ASSURES INTERFACE COORDINATION 

PROVIDES ORDERLY INTRODUCTION OF CHANGE 
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